[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:29:33 dist-test-slave-5bn2 krb5kdc[7450](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:29:33 dist-test-slave-5bn2 krb5kdc[7450](info): set up 2 sockets
Aug 06 18:29:33 dist-test-slave-5bn2 krb5kdc[7450](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 18:29:34.483117 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.317s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:29:34 dist-test-slave-5bn2 krb5kdc[7450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:29:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:29:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:29:34.610484 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:41181
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:42619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:41181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:34.725237 7467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:34.725510 7467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:34.725564 7467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:34.729481 7467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:29:34.729543 7467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:34.729568 7467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:34.729588 7467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:29:34.729607 7467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:29:34.734833 7467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:41181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:41181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:34.735898 7467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:34.737000 7467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:29:34.745600 7467 server_base.cc:1034] running on GCE node
W20260806 18:29:34.747113 7472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:34.747279 7475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:34.747608 7473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:34.748220 7467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:34.749368 7467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:34.750536 7467 hybrid_clock.cc:648] HybridClock initialized: now 1786040974750515 us; error 89 us; skew 500 ppm
Aug 06 18:29:34 dist-test-slave-5bn2 krb5kdc[7450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040974, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:34.753453 7467 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:29:34.754312 7467 webserver.cc:460] Webserver started at http://127.7.67.254:43935/ using document root <none> and password file <none>
I20260806 18:29:34.754837 7467 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:34.754937 7467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:34.755131 7467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:34.757205 7467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "e90e664caaff415ab8f1f5cf0b871bf2"
format_stamp: "Formatted at 2026-08-06 18:29:34 on dist-test-slave-5bn2"
server_key: "2c462d5858b525ddfadbb05b76bb0457"
server_key_iv: "76886f53655738b9062ee9ea94677ad5"
server_key_version: "encryptionkey@0"
I20260806 18:29:34.757767 7467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "e90e664caaff415ab8f1f5cf0b871bf2"
format_stamp: "Formatted at 2026-08-06 18:29:34 on dist-test-slave-5bn2"
server_key: "2c462d5858b525ddfadbb05b76bb0457"
server_key_iv: "76886f53655738b9062ee9ea94677ad5"
server_key_version: "encryptionkey@0"
I20260806 18:29:34.761467 7467 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260806 18:29:34.763870 7484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:34.765087 7467 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:29:34.765198 7467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "e90e664caaff415ab8f1f5cf0b871bf2"
format_stamp: "Formatted at 2026-08-06 18:29:34 on dist-test-slave-5bn2"
server_key: "2c462d5858b525ddfadbb05b76bb0457"
server_key_iv: "76886f53655738b9062ee9ea94677ad5"
server_key_version: "encryptionkey@0"
I20260806 18:29:34.765297 7467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:34.779639 7467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:34.783102 7467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:34.783293 7467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:34.790504 7467 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:41181
I20260806 18:29:34.790532 7554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:41181 every 8 connection(s)
I20260806 18:29:34.791512 7467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:29:34.794353 7557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:34.798774 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 7467
I20260806 18:29:34.798995 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:29:34.800727 7439 external_mini_cluster.cc:1428] Setting key 066c0772729f0ff7d0f19a715c912e7d
I20260806 18:29:34.800850 7557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2: Bootstrap starting.
I20260806 18:29:34.804003 7557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:34.804893 7557 log.cc:824] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:29:34.807351 7557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2: No bootstrap required, opened a new log
I20260806 18:29:34.812148 7557 raft_consensus.cc:348] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e90e664caaff415ab8f1f5cf0b871bf2" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 41181 } }
I20260806 18:29:34.812410 7557 raft_consensus.cc:374] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:34.812472 7557 raft_consensus.cc:729] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90e664caaff415ab8f1f5cf0b871bf2, State: Initialized, Role: FOLLOWER
I20260806 18:29:34.812983 7557 consensus_queue.cc:260] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e90e664caaff415ab8f1f5cf0b871bf2" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 41181 } }
I20260806 18:29:34.813146 7557 raft_consensus.cc:388] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:29:34.813213 7557 raft_consensus.cc:482] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:29:34.813315 7557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:34.814574 7557 raft_consensus.cc:504] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e90e664caaff415ab8f1f5cf0b871bf2" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 41181 } }
I20260806 18:29:34.814989 7557 leader_election.cc:302] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e90e664caaff415ab8f1f5cf0b871bf2; no voters:
I20260806 18:29:34.815346 7557 leader_election.cc:288] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:29:34.815435 7562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:34.815652 7562 raft_consensus.cc:686] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2 [term 1 LEADER]: Becoming Leader. State: Replica: e90e664caaff415ab8f1f5cf0b871bf2, State: Running, Role: LEADER
I20260806 18:29:34.815963 7562 consensus_queue.cc:237] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e90e664caaff415ab8f1f5cf0b871bf2" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 41181 } }
I20260806 18:29:34.816406 7557 sys_catalog.cc:564] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:29:34.817164 7562 sys_catalog.cc:455] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e90e664caaff415ab8f1f5cf0b871bf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e90e664caaff415ab8f1f5cf0b871bf2" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 41181 } } }
I20260806 18:29:34.817286 7562 sys_catalog.cc:458] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:29:34.817457 7564 sys_catalog.cc:455] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e90e664caaff415ab8f1f5cf0b871bf2. Latest consensus state: current_term: 1 leader_uuid: "e90e664caaff415ab8f1f5cf0b871bf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e90e664caaff415ab8f1f5cf0b871bf2" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 41181 } } }
I20260806 18:29:34.817616 7564 sys_catalog.cc:458] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:29:34.818079 7574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:29:34.821686 7574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 18:29:34 dist-test-slave-5bn2 krb5kdc[7450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:34.828459 7574 catalog_manager.cc:1347] Generated new cluster ID: bbf21313af4a4516b743cdf509d9618c
I20260806 18:29:34.828532 7574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:29:34.829824 7559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:34.804675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55746 (local address 127.7.67.254:41181)
0806 18:29:34.805204 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:34.805224 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:34.806407 (+ 1183us) server_negotiation.cc:407] Connection header received
0806 18:29:34.811730 (+ 5323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:34.811747 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:34.812065 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:34.812342 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:34.815454 (+ 3112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:34.816961 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:34.818710 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:34.819460 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:34.819642 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:34.823527 (+ 3885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:34.823533 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:34.823545 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:34.823573 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:34.825681 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:34.826264 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:34.826270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:34.826276 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:34.826457 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:34.826925 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:34.826928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:34.826930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:34.827497 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:34.827664 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:34.828445 (+ 781us) server_negotiation.cc:299] Negotiation successful
0806 18:29:34.828820 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":160,"threads_started":1}
I20260806 18:29:34.849709 7574 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:29:34.850695 7574 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:29:34.855804 7574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e90e664caaff415ab8f1f5cf0b871bf2: Generated new TSK 0
I20260806 18:29:34.856474 7574 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
I20260806 18:29:34.897305 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:41181
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:42619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:35.020156 7591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:35.020426 7591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:35.020468 7591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:35.024554 7591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:35.024622 7591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:35.024694 7591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:29:35.029975 7591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:41181
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:35.031121 7591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:35.032192 7591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:35.042194 7599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:35.042199 7600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:35.042290 7591 server_base.cc:1034] running on GCE node
W20260806 18:29:35.042194 7603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:35.042917 7591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:35.043560 7591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:35.044727 7591 hybrid_clock.cc:648] HybridClock initialized: now 1786040975044709 us; error 32 us; skew 500 ppm
Aug 06 18:29:35 dist-test-slave-5bn2 krb5kdc[7450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040975, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:35.047519 7591 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:29:35.048452 7591 webserver.cc:460] Webserver started at http://127.7.67.193:32819/ using document root <none> and password file <none>
I20260806 18:29:35.049024 7591 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:35.049108 7591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:35.049324 7591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:35.051574 7591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "40e86da33d4d4d2ca3595c3e22740083"
format_stamp: "Formatted at 2026-08-06 18:29:35 on dist-test-slave-5bn2"
server_key: "4a08f6974526a3e327634281dcfd446a"
server_key_iv: "4b3b8238468e145d7070681e938f77b4"
server_key_version: "encryptionkey@0"
I20260806 18:29:35.052192 7591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "40e86da33d4d4d2ca3595c3e22740083"
format_stamp: "Formatted at 2026-08-06 18:29:35 on dist-test-slave-5bn2"
server_key: "4a08f6974526a3e327634281dcfd446a"
server_key_iv: "4b3b8238468e145d7070681e938f77b4"
server_key_version: "encryptionkey@0"
I20260806 18:29:35.056450 7591 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:29:35.059140 7613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:35.060370 7591 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:29:35.060489 7591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "40e86da33d4d4d2ca3595c3e22740083"
format_stamp: "Formatted at 2026-08-06 18:29:35 on dist-test-slave-5bn2"
server_key: "4a08f6974526a3e327634281dcfd446a"
server_key_iv: "4b3b8238468e145d7070681e938f77b4"
server_key_version: "encryptionkey@0"
I20260806 18:29:35.060597 7591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:35.075994 7591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:35.079047 7591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:35.079252 7591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:35.079876 7591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:35.080761 7591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:35.080812 7591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:35.080869 7591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:35.080897 7591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:35.091902 7591 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:37841
I20260806 18:29:35.091928 7787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:37841 every 8 connection(s)
I20260806 18:29:35.092829 7591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:29:35.095111 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 7591
I20260806 18:29:35.095206 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:29:35.095458 7439 external_mini_cluster.cc:1428] Setting key 6022dcbd6f0c89c90d4968abf6d76e40
Aug 06 18:29:35 dist-test-slave-5bn2 krb5kdc[7450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040975, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.194@KRBTEST.COM; defaulting to no policy
I20260806 18:29:35.108295 7559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:35.094818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:60563 (local address 127.7.67.254:41181)
0806 18:29:35.095005 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:35.095010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:35.096490 (+ 1480us) server_negotiation.cc:407] Connection header received
0806 18:29:35.097543 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:35.097546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:35.097593 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:35.097660 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:35.099224 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:35.099744 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:35.101275 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:35.101590 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:35.101691 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:35.104259 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:35.104265 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:35.104268 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:35.104299 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:35.105992 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:35.106585 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:35.106589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:35.106591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:35.106688 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:35.107089 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:35.107092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:35.107093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:35.107290 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:35.107381 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:35.107973 (+ 592us) server_negotiation.cc:299] Negotiation successful
0806 18:29:35.108124 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 18:29:35.109125 7792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:35.095244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:41181 (local address 127.7.67.193:60563)
0806 18:29:35.096354 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:35.096398 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:35.097326 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:35.097860 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:35.097869 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:35.098363 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:35.099087 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:35.099108 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:35.099868 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:35.099872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:35.101159 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:35.101171 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:35.101770 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:35.101773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:35.101942 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:35.102513 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:35.102530 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:35.104072 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:35.106120 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:35.106126 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:35.106139 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:35.106448 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:35.106783 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:35.106785 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:35.106787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:35.106981 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:35.107415 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:35.107422 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:35.107679 (+ 257us) client_negotiation.cc:769] Sending connection context
0806 18:29:35.107847 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 18:29:35.108191 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"thread_start_us":693,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.194@KRBTEST.COM" created.
I20260806 18:29:35.110556 7788 heartbeater.cc:347] Connected to a master server at 127.7.67.254:41181
I20260806 18:29:35.110810 7788 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:35.111375 7788 heartbeater.cc:511] Master 127.7.67.254:41181 requested a full tablet report, sending...
I20260806 18:29:35.112943 7508 ts_manager.cc:194] Registered new tserver with Master: 40e86da33d4d4d2ca3595c3e22740083 (127.7.67.193:37841)
I20260806 18:29:35.114465 7508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:60563
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
I20260806 18:29:35.126188 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:41181
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:42619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:35.239832 7795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:35.240119 7795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:35.240170 7795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:35.244273 7795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:35.244346 7795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:35.244414 7795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:29:35.249791 7795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:41181
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:35.251004 7795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:35.252066 7795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:35.261893 7809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:35.261893 7801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:35.262081 7802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:35.261947 7795 server_base.cc:1034] running on GCE node
I20260806 18:29:35.262553 7795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:35.263253 7795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:35.264422 7795 hybrid_clock.cc:648] HybridClock initialized: now 1786040975264388 us; error 43 us; skew 500 ppm
Aug 06 18:29:35 dist-test-slave-5bn2 krb5kdc[7450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040975, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:35.267172 7795 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:29:35.268024 7795 webserver.cc:460] Webserver started at http://127.7.67.194:44373/ using document root <none> and password file <none>
I20260806 18:29:35.268589 7795 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:35.268666 7795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:35.268885 7795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:35.271096 7795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "919e103a64dc4a4cabb495e17d8db918"
format_stamp: "Formatted at 2026-08-06 18:29:35 on dist-test-slave-5bn2"
server_key: "66394140718d060f91686f15dde15dc0"
server_key_iv: "0abc7ece94faee9eb73ce821c23b09fb"
server_key_version: "encryptionkey@0"
I20260806 18:29:35.271649 7795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "919e103a64dc4a4cabb495e17d8db918"
format_stamp: "Formatted at 2026-08-06 18:29:35 on dist-test-slave-5bn2"
server_key: "66394140718d060f91686f15dde15dc0"
server_key_iv: "0abc7ece94faee9eb73ce821c23b09fb"
server_key_version: "encryptionkey@0"
I20260806 18:29:35.275398 7795 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:29:35.277835 7817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:35.279088 7795 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:29:35.279198 7795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "919e103a64dc4a4cabb495e17d8db918"
format_stamp: "Formatted at 2026-08-06 18:29:35 on dist-test-slave-5bn2"
server_key: "66394140718d060f91686f15dde15dc0"
server_key_iv: "0abc7ece94faee9eb73ce821c23b09fb"
server_key_version: "encryptionkey@0"
I20260806 18:29:35.279296 7795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:35.291927 7795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:35.295303 7795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:35.295496 7795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:35.296109 7795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:35.296960 7795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:35.297019 7795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:35.297091 7795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:35.297120 7795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:35.307161 7795 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:35817
I20260806 18:29:35.307212 7983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:35817 every 8 connection(s)
I20260806 18:29:35.308250 7795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:29:35.312858 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 7795
I20260806 18:29:35.313004 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:29:35.313274 7439 external_mini_cluster.cc:1428] Setting key 4c136b6a5ba72c25bb42453ff7cb77ea
Aug 06 18:29:35 dist-test-slave-5bn2 krb5kdc[7450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040975, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:35.323599 7559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:35.310164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:47353 (local address 127.7.67.254:41181)
0806 18:29:35.310316 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:35.310321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:35.311346 (+ 1025us) server_negotiation.cc:407] Connection header received
0806 18:29:35.312893 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:35.312896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:35.312951 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:35.313049 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:35.314358 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:35.314958 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:35.316281 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:35.316538 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:35.316636 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:35.319854 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:35.319858 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:35.319860 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:35.319887 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:35.321274 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:35.321934 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:35.321938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:35.321939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:35.322028 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:35.322410 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:35.322412 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:35.322413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:35.322592 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:35.322664 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:35.323354 (+ 690us) server_negotiation.cc:299] Negotiation successful
0806 18:29:35.323471 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.7.67.195@KRBTEST.COM; defaulting to no policy
I20260806 18:29:35.324788 7987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:35.310594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:41181 (local address 127.7.67.194:47353)
0806 18:29:35.311178 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:35.311227 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:35.312647 (+ 1420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:35.313252 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:35.313259 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:35.313659 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:35.314194 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:35.314211 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:35.315099 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:35.315102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:35.316025 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:35.316035 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:35.316756 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:35.316761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:35.316944 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:35.317624 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:35.317647 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:35.319639 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:35.321403 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:35.321411 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:35.321429 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:35.321808 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:35.322124 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:35.322128 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:35.322130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:35.322325 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:35.322693 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:35.322701 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:35.323076 (+ 375us) client_negotiation.cc:769] Sending connection context
0806 18:29:35.323312 (+ 236us) client_negotiation.cc:240] Negotiation successful
0806 18:29:35.323714 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":217,"threads_started":1}
I20260806 18:29:35.326012 7984 heartbeater.cc:347] Connected to a master server at 127.7.67.254:41181
I20260806 18:29:35.326270 7984 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:35.326802 7984 heartbeater.cc:511] Master 127.7.67.254:41181 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.195@KRBTEST.COM" created.
I20260806 18:29:35.328090 7508 ts_manager.cc:194] Registered new tserver with Master: 919e103a64dc4a4cabb495e17d8db918 (127.7.67.194:35817)
I20260806 18:29:35.328892 7508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:47353
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
I20260806 18:29:35.345428 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:41181
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:42619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:35.459842 7991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:35.460103 7991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:35.460162 7991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:35.464205 7991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:35.464269 7991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:35.464347 7991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:29:35.469765 7991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:41181
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:35.470942 7991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:35.471979 7991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:35.481606 7999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:35.481588 7996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:35.481793 7991 server_base.cc:1034] running on GCE node
W20260806 18:29:35.481588 7997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:35.482263 7991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:35.482931 7991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:35.484112 7991 hybrid_clock.cc:648] HybridClock initialized: now 1786040975484094 us; error 31 us; skew 500 ppm
Aug 06 18:29:35 dist-test-slave-5bn2 krb5kdc[7450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040975, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:35.486799 7991 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:29:35.487717 7991 webserver.cc:460] Webserver started at http://127.7.67.195:42625/ using document root <none> and password file <none>
I20260806 18:29:35.488267 7991 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:35.488335 7991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:35.488531 7991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:35.490646 7991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "116d5c9a88c844148fcebe63e0a8eebd"
format_stamp: "Formatted at 2026-08-06 18:29:35 on dist-test-slave-5bn2"
server_key: "b075cf1c3ea274809f735cc1602ebf46"
server_key_iv: "13f2d1c224160a18a222acb7ca60327d"
server_key_version: "encryptionkey@0"
I20260806 18:29:35.491240 7991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "116d5c9a88c844148fcebe63e0a8eebd"
format_stamp: "Formatted at 2026-08-06 18:29:35 on dist-test-slave-5bn2"
server_key: "b075cf1c3ea274809f735cc1602ebf46"
server_key_iv: "13f2d1c224160a18a222acb7ca60327d"
server_key_version: "encryptionkey@0"
I20260806 18:29:35.495049 7991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:29:35.497406 8008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:35.498543 7991 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:29:35.498667 7991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "116d5c9a88c844148fcebe63e0a8eebd"
format_stamp: "Formatted at 2026-08-06 18:29:35 on dist-test-slave-5bn2"
server_key: "b075cf1c3ea274809f735cc1602ebf46"
server_key_iv: "13f2d1c224160a18a222acb7ca60327d"
server_key_version: "encryptionkey@0"
I20260806 18:29:35.498760 7991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:35.512120 7991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:35.515244 7991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:35.515426 7991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:35.516026 7991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:35.516902 7991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:35.516950 7991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:35.517011 7991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:35.517040 7991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:35.527432 7991 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:38243
I20260806 18:29:35.527457 8174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:38243 every 8 connection(s)
I20260806 18:29:35.528313 7991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:29:35.532229 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 7991
I20260806 18:29:35.532344 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:29:35.532600 7439 external_mini_cluster.cc:1428] Setting key 9a5fe53614885eaab55976eb4a04956c
Aug 06 18:29:35 dist-test-slave-5bn2 krb5kdc[7450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040975, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:35.541612 7559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:35.530252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:43565 (local address 127.7.67.254:41181)
0806 18:29:35.530383 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:35.530387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:35.531278 (+ 891us) server_negotiation.cc:407] Connection header received
0806 18:29:35.532646 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:35.532650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:35.532710 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:35.532784 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:35.534124 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:35.534878 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:35.536072 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:35.536304 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:35.536417 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:35.538756 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:35.538759 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:35.538761 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:35.538786 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:35.539945 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:35.540391 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:35.540393 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:35.540394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:35.540460 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:35.540714 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:35.540716 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:35.540717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:35.540884 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:35.540953 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:35.541387 (+ 434us) server_negotiation.cc:299] Negotiation successful
0806 18:29:35.541483 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:29:35.542418 8178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:35.530632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:41181 (local address 127.7.67.195:43565)
0806 18:29:35.531135 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:35.531179 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:35.532434 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:35.532973 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:35.532979 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:35.533420 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:35.533973 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:35.533989 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:35.535024 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:35.535028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:35.535937 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:35.535948 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:35.536448 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:35.536450 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:35.536603 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:35.537165 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:35.537182 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:35.538598 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:35.540046 (+ 1448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:35.540051 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:35.540063 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:35.540312 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:35.540522 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:35.540523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:35.540525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:35.540652 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:35.540952 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:35.540957 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:35.541201 (+ 244us) client_negotiation.cc:769] Sending connection context
0806 18:29:35.541355 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 18:29:35.541605 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":129,"threads_started":1}
I20260806 18:29:35.543551 8175 heartbeater.cc:347] Connected to a master server at 127.7.67.254:41181
I20260806 18:29:35.543794 8175 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:35.544298 8175 heartbeater.cc:511] Master 127.7.67.254:41181 requested a full tablet report, sending...
I20260806 18:29:35.545336 7508 ts_manager.cc:194] Registered new tserver with Master: 116d5c9a88c844148fcebe63e0a8eebd (127.7.67.195:38243)
I20260806 18:29:35.545972 7508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:43565
I20260806 18:29:35.547035 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:29:35 dist-test-slave-5bn2 krb5kdc[7450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040975, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:29:35 dist-test-slave-5bn2 krb5kdc[7450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040975, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.67.193@KRBTEST.COM
I20260806 18:29:35.574028 8189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:35.564794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43360 (local address 127.7.67.193:37841)
0806 18:29:35.565057 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:35.565073 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:35.565101 (+ 28us) server_negotiation.cc:407] Connection header received
0806 18:29:35.565256 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:35.565264 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:35.565479 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:35.565637 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:35.566321 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:35.567171 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:35.567988 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:35.568390 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:35.568486 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:35.570826 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:35.570837 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:35.570875 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:35.570902 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:35.572302 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:35.572699 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:35.572705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:35.572707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:35.572789 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:35.573037 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:35.573040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:35.573042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:35.573291 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:35.573527 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:35.573684 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 18:29:35.573837 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":64,"threads_started":1}
W20260806 18:29:35.575063 7657 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43360
Aug 06 18:29:35 dist-test-slave-5bn2 krb5kdc[7450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:29:35 dist-test-slave-5bn2 krb5kdc[7450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.193@KRBTEST.COM
I20260806 18:29:35.602459 8189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:35.594628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43366 (local address 127.7.67.193:37841)
0806 18:29:35.594816 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:35.594819 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:35.594923 (+ 104us) server_negotiation.cc:407] Connection header received
0806 18:29:35.595050 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:35.595052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:35.595098 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:35.595168 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:35.595837 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:35.596357 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:35.597111 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:35.597322 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:35.597388 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:35.599706 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:35.599710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:35.599712 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:35.599739 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:35.600869 (+ 1130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:35.601264 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:35.601267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:35.601268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:35.601349 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:35.601626 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:35.601629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:35.601631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:35.601810 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:35.601930 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:35.602166 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 18:29:35.602292 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 18:29:35.603571 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 7591
I20260806 18:29:35.616174 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 7795
I20260806 18:29:35.627612 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 7991
I20260806 18:29:35.638123 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 7467
2026-08-06T18:29:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2509 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:29:35 dist-test-slave-5bn2 krb5kdc[8198](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:29:35 dist-test-slave-5bn2 krb5kdc[8198](info): set up 2 sockets
Aug 06 18:29:35 dist-test-slave-5bn2 krb5kdc[8198](info): commencing operation
krb5kdc: starting...
W20260806 18:29:36.749186 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:29:36 dist-test-slave-5bn2 krb5kdc[8198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:29:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:29:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:29:36.871464 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:37863
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:43673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:37863
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:36.983321 8215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:36.983593 8215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:36.983650 8215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:36.987594 8215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:29:36.987654 8215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:36.987682 8215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:36.987701 8215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:29:36.987722 8215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:29:36.993005 8215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:43673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:37863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:37863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:36.994144 8215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:36.995251 8215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:37.003782 8222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:37.003867 8215 server_base.cc:1034] running on GCE node
W20260806 18:29:37.003810 8221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:37.003799 8224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:37.004655 8215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:37.005751 8215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:37.006922 8215 hybrid_clock.cc:648] HybridClock initialized: now 1786040977006896 us; error 37 us; skew 500 ppm
Aug 06 18:29:37 dist-test-slave-5bn2 krb5kdc[8198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040977, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:37.009711 8215 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:29:37.010674 8215 webserver.cc:460] Webserver started at http://127.7.67.254:44171/ using document root <none> and password file <none>
I20260806 18:29:37.011264 8215 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:37.011334 8215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:37.011544 8215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:37.013604 8215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "66f3255cd2e143518b4e97eb3acb99ba"
format_stamp: "Formatted at 2026-08-06 18:29:37 on dist-test-slave-5bn2"
server_key: "261fc7adbd37d35a172769f2a6d09db7"
server_key_iv: "2a7d2d0715b3d2dd0774715f0a1eceef"
server_key_version: "encryptionkey@0"
I20260806 18:29:37.014168 8215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "66f3255cd2e143518b4e97eb3acb99ba"
format_stamp: "Formatted at 2026-08-06 18:29:37 on dist-test-slave-5bn2"
server_key: "261fc7adbd37d35a172769f2a6d09db7"
server_key_iv: "2a7d2d0715b3d2dd0774715f0a1eceef"
server_key_version: "encryptionkey@0"
I20260806 18:29:37.017993 8215 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:29:37.020525 8239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:37.021713 8215 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:29:37.021832 8215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "66f3255cd2e143518b4e97eb3acb99ba"
format_stamp: "Formatted at 2026-08-06 18:29:37 on dist-test-slave-5bn2"
server_key: "261fc7adbd37d35a172769f2a6d09db7"
server_key_iv: "2a7d2d0715b3d2dd0774715f0a1eceef"
server_key_version: "encryptionkey@0"
I20260806 18:29:37.021929 8215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:37.043923 8215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:37.047282 8215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:37.047474 8215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:37.054570 8215 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:37863
I20260806 18:29:37.054596 8318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:37863 every 8 connection(s)
I20260806 18:29:37.055577 8215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:29:37.057791 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 8215
I20260806 18:29:37.057910 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:29:37.058205 7439 external_mini_cluster.cc:1428] Setting key 0c35ed87971df9703d0d43d88cfab79d
I20260806 18:29:37.058740 8319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:37.065687 8319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba: Bootstrap starting.
Aug 06 18:29:37 dist-test-slave-5bn2 krb5kdc[8198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:37.068410 8319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:37.069125 8319 log.cc:824] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba: Log is configured to *not* fsync() on all Append() calls
I20260806 18:29:37.071424 8319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba: No bootstrap required, opened a new log
I20260806 18:29:37.072624 8325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.059692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.7.67.254:37863)
0806 18:29:37.060124 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:37.060140 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:37.060177 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:29:37.060862 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:37.060879 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:37.061235 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:37.061520 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:37.062413 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.063490 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.064623 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.065088 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.065198 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:37.067963 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:37.067968 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:37.067981 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:37.068012 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:37.069765 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:37.070266 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:37.070271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:37.070276 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:37.070390 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:37.070649 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:37.070651 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:37.070652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:37.071111 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:37.071265 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:37.071565 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 18:29:37.071874 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":123,"threads_started":1}
I20260806 18:29:37.075214 8319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66f3255cd2e143518b4e97eb3acb99ba" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 37863 } }
I20260806 18:29:37.075436 8319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:37.075495 8319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66f3255cd2e143518b4e97eb3acb99ba, State: Initialized, Role: FOLLOWER
I20260806 18:29:37.075891 8319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66f3255cd2e143518b4e97eb3acb99ba" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 37863 } }
I20260806 18:29:37.076016 8319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:29:37.076078 8319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:29:37.076181 8319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:37.077185 8319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66f3255cd2e143518b4e97eb3acb99ba" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 37863 } }
I20260806 18:29:37.077497 8319 leader_election.cc:302] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66f3255cd2e143518b4e97eb3acb99ba; no voters:
I20260806 18:29:37.077888 8319 leader_election.cc:288] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:29:37.077997 8327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:37.078233 8327 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba [term 1 LEADER]: Becoming Leader. State: Replica: 66f3255cd2e143518b4e97eb3acb99ba, State: Running, Role: LEADER
I20260806 18:29:37.078580 8327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66f3255cd2e143518b4e97eb3acb99ba" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 37863 } }
I20260806 18:29:37.079109 8319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:29:37.080271 8328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66f3255cd2e143518b4e97eb3acb99ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66f3255cd2e143518b4e97eb3acb99ba" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 37863 } } }
I20260806 18:29:37.080413 8328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba [sys.catalog]: This master's current role is: LEADER
I20260806 18:29:37.080798 8343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:29:37.081135 8329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66f3255cd2e143518b4e97eb3acb99ba. Latest consensus state: current_term: 1 leader_uuid: "66f3255cd2e143518b4e97eb3acb99ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66f3255cd2e143518b4e97eb3acb99ba" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 37863 } } }
I20260806 18:29:37.081235 8329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba [sys.catalog]: This master's current role is: LEADER
I20260806 18:29:37.083988 8343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:29:37.089663 8343 catalog_manager.cc:1347] Generated new cluster ID: 69f045e097a34f03b8ef02de8b1921bf
I20260806 18:29:37.089735 8343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:29:37.098307 8343 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:29:37.099288 8343 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:29:37.104271 8343 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66f3255cd2e143518b4e97eb3acb99ba: Generated new TSK 0
I20260806 18:29:37.105082 8343 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
I20260806 18:29:37.141230 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:37863
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:43673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:37.255806 8367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:37.256068 8367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:37.256110 8367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:37.260088 8367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:37.260146 8367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:37.260206 8367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:29:37.265394 8367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:43673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:37863
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:37.266424 8367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:37.267518 8367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:29:37.277333 8367 server_base.cc:1034] running on GCE node
W20260806 18:29:37.278913 8376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:37.279510 8377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:37.279717 8379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:37.280177 8367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:37.280880 8367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:37.282063 8367 hybrid_clock.cc:648] HybridClock initialized: now 1786040977282032 us; error 43 us; skew 500 ppm
Aug 06 18:29:37 dist-test-slave-5bn2 krb5kdc[8198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040977, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:37.288307 8367 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:29:37.289268 8367 webserver.cc:460] Webserver started at http://127.7.67.193:35793/ using document root <none> and password file <none>
I20260806 18:29:37.289872 8367 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:37.289939 8367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:37.290166 8367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:37.292331 8367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "d4bff7b1830c4b2fa96c9049e1f0fee9"
format_stamp: "Formatted at 2026-08-06 18:29:37 on dist-test-slave-5bn2"
server_key: "db4c808a47a50d67246121cc11b9607b"
server_key_iv: "09788f25c1b437433a81757302e2c7dd"
server_key_version: "encryptionkey@0"
I20260806 18:29:37.292894 8367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "d4bff7b1830c4b2fa96c9049e1f0fee9"
format_stamp: "Formatted at 2026-08-06 18:29:37 on dist-test-slave-5bn2"
server_key: "db4c808a47a50d67246121cc11b9607b"
server_key_iv: "09788f25c1b437433a81757302e2c7dd"
server_key_version: "encryptionkey@0"
I20260806 18:29:37.296937 8367 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:29:37.299539 8386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:37.300807 8367 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:29:37.300928 8367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "d4bff7b1830c4b2fa96c9049e1f0fee9"
format_stamp: "Formatted at 2026-08-06 18:29:37 on dist-test-slave-5bn2"
server_key: "db4c808a47a50d67246121cc11b9607b"
server_key_iv: "09788f25c1b437433a81757302e2c7dd"
server_key_version: "encryptionkey@0"
I20260806 18:29:37.301046 8367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:37.320281 8367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:37.323102 8367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:37.323285 8367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:37.323891 8367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:37.324764 8367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:37.324820 8367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:37.324882 8367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:37.324908 8367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:37.333324 8367 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:41379
I20260806 18:29:37.333350 8555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:41379 every 8 connection(s)
I20260806 18:29:37.334223 8367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:29:37.340237 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 8367
I20260806 18:29:37.340359 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:29:37.340639 7439 external_mini_cluster.cc:1428] Setting key f166aaa06d8f274d0e4b0be63b934a51
Aug 06 18:29:37 dist-test-slave-5bn2 krb5kdc[8198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040977, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:37.348361 8325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.336061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:43893 (local address 127.7.67.254:37863)
0806 18:29:37.336196 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:37.336200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:37.337032 (+ 832us) server_negotiation.cc:407] Connection header received
0806 18:29:37.338406 (+ 1374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:37.338410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:37.338472 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:37.338544 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:37.339796 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.340506 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.341932 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.342220 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.342324 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:37.344889 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:37.344892 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:37.344893 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:37.344917 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:37.346226 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:37.346766 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:37.346770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:37.346772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:37.346899 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:37.347216 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:37.347219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:37.347221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:37.347452 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:37.347571 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:37.348028 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 18:29:37.348183 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:29:37.349088 8561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.336427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:37863 (local address 127.7.67.193:43893)
0806 18:29:37.336878 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:37.336921 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:37.338185 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:37.338736 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:37.338743 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:37.339147 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:37.339640 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:37.339655 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.340655 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.340658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:37.341804 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:37.341815 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.342384 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.342387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:37.342538 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:37.343118 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:37.343134 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:37.344731 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:37.346338 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:37.346344 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:37.346359 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:37.346641 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:37.346986 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:37.346989 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:37.346990 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:37.347129 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:37.347555 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:37.347568 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:37.347802 (+ 234us) client_negotiation.cc:769] Sending connection context
0806 18:29:37.347955 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 18:29:37.348214 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":118,"threads_started":1}
I20260806 18:29:37.350327 8557 heartbeater.cc:347] Connected to a master server at 127.7.67.254:37863
I20260806 18:29:37.350574 8557 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:37.351111 8557 heartbeater.cc:511] Master 127.7.67.254:37863 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.67.194@KRBTEST.COM; defaulting to no policy
I20260806 18:29:37.352645 8263 ts_manager.cc:194] Registered new tserver with Master: d4bff7b1830c4b2fa96c9049e1f0fee9 (127.7.67.193:41379)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.194@KRBTEST.COM" created.
I20260806 18:29:37.353809 8263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:43893
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
I20260806 18:29:37.367717 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:37863
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:43673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:37.498499 8566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:37.498792 8566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:37.498881 8566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:37.502842 8566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:37.502934 8566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:37.503021 8566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:29:37.508380 8566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:43673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:37863
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:37.509495 8566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:37.510605 8566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:37.520423 8575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:37.520442 8576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:37.520727 8566 server_base.cc:1034] running on GCE node
W20260806 18:29:37.520442 8580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:37.521166 8566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:37.521814 8566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:37.522957 8566 hybrid_clock.cc:648] HybridClock initialized: now 1786040977522923 us; error 47 us; skew 500 ppm
Aug 06 18:29:37 dist-test-slave-5bn2 krb5kdc[8198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040977, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:37.525708 8566 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:29:37.526625 8566 webserver.cc:460] Webserver started at http://127.7.67.194:36287/ using document root <none> and password file <none>
I20260806 18:29:37.527231 8566 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:37.527302 8566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:37.527511 8566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:37.529583 8566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "c709f45ee1954c4da2719309fc47b9e5"
format_stamp: "Formatted at 2026-08-06 18:29:37 on dist-test-slave-5bn2"
server_key: "a97231490ebf74133469b1f81e39d87d"
server_key_iv: "31a341ad96fa52904d17d8419df4d55b"
server_key_version: "encryptionkey@0"
I20260806 18:29:37.530164 8566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "c709f45ee1954c4da2719309fc47b9e5"
format_stamp: "Formatted at 2026-08-06 18:29:37 on dist-test-slave-5bn2"
server_key: "a97231490ebf74133469b1f81e39d87d"
server_key_iv: "31a341ad96fa52904d17d8419df4d55b"
server_key_version: "encryptionkey@0"
I20260806 18:29:37.533869 8566 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:29:37.536363 8595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:37.537568 8566 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:29:37.537684 8566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "c709f45ee1954c4da2719309fc47b9e5"
format_stamp: "Formatted at 2026-08-06 18:29:37 on dist-test-slave-5bn2"
server_key: "a97231490ebf74133469b1f81e39d87d"
server_key_iv: "31a341ad96fa52904d17d8419df4d55b"
server_key_version: "encryptionkey@0"
I20260806 18:29:37.537786 8566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:37.549620 8566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:37.552870 8566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:37.553068 8566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:37.553663 8566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:37.554502 8566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:37.554551 8566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:37.554605 8566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:37.554632 8566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:37.562968 8566 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:37123
I20260806 18:29:37.562991 8757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:37123 every 8 connection(s)
I20260806 18:29:37.563853 8566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:29:37.564438 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 8566
I20260806 18:29:37.564574 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:29:37.564888 7439 external_mini_cluster.cc:1428] Setting key 83581b6324955e391e439bd23413f257
Aug 06 18:29:37 dist-test-slave-5bn2 krb5kdc[8198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040977, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.195@KRBTEST.COM" created.
I20260806 18:29:37.578927 8325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.565814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:35823 (local address 127.7.67.254:37863)
0806 18:29:37.565949 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:37.565954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:37.566969 (+ 1015us) server_negotiation.cc:407] Connection header received
0806 18:29:37.568397 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:37.568400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:37.568448 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:37.568521 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:37.570053 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.570604 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.572338 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.572566 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.572643 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:37.575397 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:37.575401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:37.575403 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:37.575429 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:37.576751 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:37.577253 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:37.577255 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:37.577257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:37.577334 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:37.577662 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:37.577666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:37.577667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:37.577880 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:37.577969 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:37.578621 (+ 652us) server_negotiation.cc:299] Negotiation successful
0806 18:29:37.578734 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:29:37.579962 8762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.566296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:37863 (local address 127.7.67.194:35823)
0806 18:29:37.566795 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:37.566870 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:37.568153 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:37.568721 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:37.568731 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:37.569221 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:37.569911 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:37.569931 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.570741 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.570745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:37.572196 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:37.572209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.572723 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.572728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:37.572938 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:37.573550 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:37.573579 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:37.575228 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:37.576854 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:37.576861 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:37.576877 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:37.577145 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:37.577414 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:37.577418 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:37.577420 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:37.577561 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:37.577981 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:37.577987 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:37.578318 (+ 331us) client_negotiation.cc:769] Sending connection context
0806 18:29:37.578526 (+ 208us) client_negotiation.cc:240] Negotiation successful
0806 18:29:37.578905 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":111,"threads_started":1}
I20260806 18:29:37.581261 8760 heartbeater.cc:347] Connected to a master server at 127.7.67.254:37863
I20260806 18:29:37.581542 8760 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:37.582094 8760 heartbeater.cc:511] Master 127.7.67.254:37863 requested a full tablet report, sending...
I20260806 18:29:37.583104 8263 ts_manager.cc:194] Registered new tserver with Master: c709f45ee1954c4da2719309fc47b9e5 (127.7.67.194:37123)
I20260806 18:29:37.583668 8263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:35823
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
I20260806 18:29:37.593698 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:37863
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:43673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:37.707681 8765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:37.707955 8765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:37.708017 8765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:37.712030 8765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:37.712090 8765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:37.712170 8765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:29:37.717410 8765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:43673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:37863
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:37.718472 8765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:37.719547 8765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:37.729250 8775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:37.729215 8774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:37.729238 8778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:37.729636 8765 server_base.cc:1034] running on GCE node
I20260806 18:29:37.730083 8765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:37.730697 8765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:37.731860 8765 hybrid_clock.cc:648] HybridClock initialized: now 1786040977731842 us; error 48 us; skew 500 ppm
Aug 06 18:29:37 dist-test-slave-5bn2 krb5kdc[8198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040977, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:37.734488 8765 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:29:37.735442 8765 webserver.cc:460] Webserver started at http://127.7.67.195:45703/ using document root <none> and password file <none>
I20260806 18:29:37.735991 8765 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:37.736070 8765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:37.736282 8765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:37.738325 8765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "8854301ae6dc48dfaaaa91f27ef72c3c"
format_stamp: "Formatted at 2026-08-06 18:29:37 on dist-test-slave-5bn2"
server_key: "4f4e7789e968ca4b6ebae8d95abee628"
server_key_iv: "91770e863510a9d0575cfee92f1ecd00"
server_key_version: "encryptionkey@0"
I20260806 18:29:37.738940 8765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "8854301ae6dc48dfaaaa91f27ef72c3c"
format_stamp: "Formatted at 2026-08-06 18:29:37 on dist-test-slave-5bn2"
server_key: "4f4e7789e968ca4b6ebae8d95abee628"
server_key_iv: "91770e863510a9d0575cfee92f1ecd00"
server_key_version: "encryptionkey@0"
I20260806 18:29:37.742689 8765 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:29:37.745313 8787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:37.746503 8765 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:29:37.746611 8765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "8854301ae6dc48dfaaaa91f27ef72c3c"
format_stamp: "Formatted at 2026-08-06 18:29:37 on dist-test-slave-5bn2"
server_key: "4f4e7789e968ca4b6ebae8d95abee628"
server_key_iv: "91770e863510a9d0575cfee92f1ecd00"
server_key_version: "encryptionkey@0"
I20260806 18:29:37.746719 8765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:37.769486 8765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:37.772845 8765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:37.773042 8765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:37.773669 8765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:37.774537 8765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:37.774595 8765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:37.774654 8765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:37.774685 8765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:37.786311 8765 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:42345
I20260806 18:29:37.786352 8965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:42345 every 8 connection(s)
I20260806 18:29:37.787293 8765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:29:37.790200 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 8765
I20260806 18:29:37.790318 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:29:37.790608 7439 external_mini_cluster.cc:1428] Setting key 65645da3c342e0614490c2f37094cc02
Aug 06 18:29:37 dist-test-slave-5bn2 krb5kdc[8198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040977, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:37.800609 8325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.789159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:37409 (local address 127.7.67.254:37863)
0806 18:29:37.789328 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:37.789333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:37.790205 (+ 872us) server_negotiation.cc:407] Connection header received
0806 18:29:37.791724 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:37.791727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:37.791784 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:37.791862 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:37.793238 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.793771 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.794812 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.795053 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.795127 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:37.797493 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:37.797496 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:37.797498 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:37.797522 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:37.798633 (+ 1111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:37.799261 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:37.799264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:37.799265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:37.799339 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:37.799605 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:37.799607 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:37.799608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:37.799765 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:37.799833 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:37.800371 (+ 538us) server_negotiation.cc:299] Negotiation successful
0806 18:29:37.800475 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:29:37.801489 8969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.789574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:37863 (local address 127.7.67.195:37409)
0806 18:29:37.790049 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:37.790105 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:37.791522 (+ 1417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:37.792054 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:37.792064 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:37.792560 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:37.793109 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:37.793126 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.793858 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.793860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:37.794720 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:37.794729 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.795169 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.795172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:37.795312 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:37.795884 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:37.795901 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:37.797335 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:37.798729 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:37.798734 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:37.798746 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:37.799185 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:37.799405 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:37.799407 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:37.799409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:37.799542 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:37.799833 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:37.799838 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:37.800117 (+ 279us) client_negotiation.cc:769] Sending connection context
0806 18:29:37.800316 (+ 199us) client_negotiation.cc:240] Negotiation successful
0806 18:29:37.800582 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":109,"threads_started":1}
I20260806 18:29:37.802646 8967 heartbeater.cc:347] Connected to a master server at 127.7.67.254:37863
I20260806 18:29:37.802964 8967 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:37.803553 8967 heartbeater.cc:511] Master 127.7.67.254:37863 requested a full tablet report, sending...
I20260806 18:29:37.804653 8263 ts_manager.cc:194] Registered new tserver with Master: 8854301ae6dc48dfaaaa91f27ef72c3c (127.7.67.195:42345)
I20260806 18:29:37.805238 8263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:37409
I20260806 18:29:37.816362 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:29:37.827807 8325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.820729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57570 (local address 127.7.67.254:37863)
0806 18:29:37.820855 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:37.820860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:37.821029 (+ 169us) server_negotiation.cc:407] Connection header received
0806 18:29:37.821167 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:37.821169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:37.821214 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:37.821275 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:37.822000 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.822530 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.823386 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.823605 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.823692 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:37.825033 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:37.825037 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:37.825039 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:37.825063 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:37.826175 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:37.826593 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:37.826596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:37.826597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:37.826675 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:37.826952 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:37.826956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:37.826958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:37.827179 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:37.827298 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:37.827540 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 18:29:37.827652 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:29:37.833143 8263 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:29:37.836200 8263 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:29:37.850329 8981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.844796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:42345 (local address 127.0.0.1:54074)
0806 18:29:37.845120 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:37.845161 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:37.845300 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:37.846042 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:37.846048 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:37.846074 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:37.846223 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:37.846230 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.847569 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.847572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:37.848435 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:37.848444 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.849742 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.849746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:37.849881 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:37.849924 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 18:29:37.850023 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 18:29:37.850177 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":93,"threads_started":1}
I20260806 18:29:37.850445 8983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.844971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54074 (local address 127.7.67.195:42345)
0806 18:29:37.845370 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:37.845383 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:37.845410 (+ 27us) server_negotiation.cc:407] Connection header received
0806 18:29:37.845483 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:37.845487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:37.845665 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:37.845823 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:37.846345 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.847455 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.848557 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.849390 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.849498 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:37.849645 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:37.850086 (+ 441us) server_negotiation.cc:299] Negotiation successful
0806 18:29:37.850190 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":173,"threads_started":1}
I20260806 18:29:37.851557 8987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.845517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:37123 (local address 127.0.0.1:42634)
0806 18:29:37.845799 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:37.845812 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:37.845898 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:37.847017 (+ 1119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:37.847021 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:37.847040 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:37.847159 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:37.847166 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.848666 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.848669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:37.849790 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:37.849798 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.851203 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.851207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:37.851299 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:37.851326 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:29:37.851398 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 18:29:37.851455 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":152,"threads_started":1}
I20260806 18:29:37.851697 8990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.845713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42634 (local address 127.7.67.194:37123)
0806 18:29:37.846534 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:37.846547 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:37.846571 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:29:37.846648 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:37.846653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:37.846839 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:37.847304 (+ 465us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:37.847476 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.848559 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.849943 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.851089 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.851190 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:37.851335 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:37.851444 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 18:29:37.851540 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720,"thread_start_us":116,"threads_started":1}
I20260806 18:29:37.853442 8866 tablet_service.cc:1467] Processing CreateTablet for tablet 21f28a11b11e4c9b911a891731e1796e (DEFAULT_TABLE table=test-table [id=69a6fcf263ca4bafaadd3c1a88e038b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:37.854206 8660 tablet_service.cc:1467] Processing CreateTablet for tablet 21f28a11b11e4c9b911a891731e1796e (DEFAULT_TABLE table=test-table [id=69a6fcf263ca4bafaadd3c1a88e038b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:37.854550 8866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21f28a11b11e4c9b911a891731e1796e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:37.855194 8660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21f28a11b11e4c9b911a891731e1796e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:37.856208 8991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.845915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:41379 (local address 127.0.0.1:40460)
0806 18:29:37.846142 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:37.846154 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:37.846233 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:37.847109 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:37.847112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:37.847133 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:37.847230 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:37.847236 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.849715 (+ 2479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.849718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:37.850838 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:37.850901 (+ 63us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.855768 (+ 4867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.855772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:37.855880 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:37.855908 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 18:29:37.855989 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 18:29:37.856071 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":98,"threads_started":1}
I20260806 18:29:37.856452 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.846320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40460 (local address 127.7.67.193:41379)
0806 18:29:37.846700 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:37.846715 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:37.846734 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:29:37.846798 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:37.846802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:37.846993 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:37.847138 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:37.847324 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.848522 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.851038 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.855609 (+ 4571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.855724 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:37.855867 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:37.856031 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 18:29:37.856146 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":77,"threads_started":1}
I20260806 18:29:37.859833 8458 tablet_service.cc:1467] Processing CreateTablet for tablet 21f28a11b11e4c9b911a891731e1796e (DEFAULT_TABLE table=test-table [id=69a6fcf263ca4bafaadd3c1a88e038b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:37.860941 8458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21f28a11b11e4c9b911a891731e1796e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:37.861696 8994 tablet_bootstrap.cc:492] T 21f28a11b11e4c9b911a891731e1796e P 8854301ae6dc48dfaaaa91f27ef72c3c: Bootstrap starting.
I20260806 18:29:37.862814 8996 tablet_bootstrap.cc:492] T 21f28a11b11e4c9b911a891731e1796e P c709f45ee1954c4da2719309fc47b9e5: Bootstrap starting.
I20260806 18:29:37.864157 8994 tablet_bootstrap.cc:654] T 21f28a11b11e4c9b911a891731e1796e P 8854301ae6dc48dfaaaa91f27ef72c3c: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:37.865496 8996 tablet_bootstrap.cc:654] T 21f28a11b11e4c9b911a891731e1796e P c709f45ee1954c4da2719309fc47b9e5: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:37.865576 8994 log.cc:824] T 21f28a11b11e4c9b911a891731e1796e P 8854301ae6dc48dfaaaa91f27ef72c3c: Log is configured to *not* fsync() on all Append() calls
I20260806 18:29:37.866264 8996 log.cc:824] T 21f28a11b11e4c9b911a891731e1796e P c709f45ee1954c4da2719309fc47b9e5: Log is configured to *not* fsync() on all Append() calls
I20260806 18:29:37.867431 9001 tablet_bootstrap.cc:492] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9: Bootstrap starting.
I20260806 18:29:37.868328 8994 tablet_bootstrap.cc:492] T 21f28a11b11e4c9b911a891731e1796e P 8854301ae6dc48dfaaaa91f27ef72c3c: No bootstrap required, opened a new log
I20260806 18:29:37.868548 8994 ts_tablet_manager.cc:1397] T 21f28a11b11e4c9b911a891731e1796e P 8854301ae6dc48dfaaaa91f27ef72c3c: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 18:29:37.868868 8996 tablet_bootstrap.cc:492] T 21f28a11b11e4c9b911a891731e1796e P c709f45ee1954c4da2719309fc47b9e5: No bootstrap required, opened a new log
I20260806 18:29:37.869065 8996 ts_tablet_manager.cc:1397] T 21f28a11b11e4c9b911a891731e1796e P c709f45ee1954c4da2719309fc47b9e5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:29:37.869417 9001 tablet_bootstrap.cc:654] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:37.870067 9001 log.cc:824] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9: Log is configured to *not* fsync() on all Append() calls
I20260806 18:29:37.871685 9001 tablet_bootstrap.cc:492] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9: No bootstrap required, opened a new log
I20260806 18:29:37.871860 9001 ts_tablet_manager.cc:1397] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:29:37.872578 8994 raft_consensus.cc:348] T 21f28a11b11e4c9b911a891731e1796e P 8854301ae6dc48dfaaaa91f27ef72c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8854301ae6dc48dfaaaa91f27ef72c3c" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 42345 } } peers { permanent_uuid: "c709f45ee1954c4da2719309fc47b9e5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 37123 } } peers { permanent_uuid: "d4bff7b1830c4b2fa96c9049e1f0fee9" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 41379 } }
I20260806 18:29:37.872885 8994 raft_consensus.cc:374] T 21f28a11b11e4c9b911a891731e1796e P 8854301ae6dc48dfaaaa91f27ef72c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:37.872942 8994 raft_consensus.cc:729] T 21f28a11b11e4c9b911a891731e1796e P 8854301ae6dc48dfaaaa91f27ef72c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8854301ae6dc48dfaaaa91f27ef72c3c, State: Initialized, Role: FOLLOWER
I20260806 18:29:37.872797 8996 raft_consensus.cc:348] T 21f28a11b11e4c9b911a891731e1796e P c709f45ee1954c4da2719309fc47b9e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8854301ae6dc48dfaaaa91f27ef72c3c" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 42345 } } peers { permanent_uuid: "c709f45ee1954c4da2719309fc47b9e5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 37123 } } peers { permanent_uuid: "d4bff7b1830c4b2fa96c9049e1f0fee9" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 41379 } }
I20260806 18:29:37.873066 8996 raft_consensus.cc:374] T 21f28a11b11e4c9b911a891731e1796e P c709f45ee1954c4da2719309fc47b9e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:37.873117 8996 raft_consensus.cc:729] T 21f28a11b11e4c9b911a891731e1796e P c709f45ee1954c4da2719309fc47b9e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c709f45ee1954c4da2719309fc47b9e5, State: Initialized, Role: FOLLOWER
I20260806 18:29:37.873417 8994 consensus_queue.cc:260] T 21f28a11b11e4c9b911a891731e1796e P 8854301ae6dc48dfaaaa91f27ef72c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8854301ae6dc48dfaaaa91f27ef72c3c" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 42345 } } peers { permanent_uuid: "c709f45ee1954c4da2719309fc47b9e5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 37123 } } peers { permanent_uuid: "d4bff7b1830c4b2fa96c9049e1f0fee9" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 41379 } }
I20260806 18:29:37.873452 8996 consensus_queue.cc:260] T 21f28a11b11e4c9b911a891731e1796e P c709f45ee1954c4da2719309fc47b9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8854301ae6dc48dfaaaa91f27ef72c3c" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 42345 } } peers { permanent_uuid: "c709f45ee1954c4da2719309fc47b9e5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 37123 } } peers { permanent_uuid: "d4bff7b1830c4b2fa96c9049e1f0fee9" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 41379 } }
I20260806 18:29:37.874118 8967 heartbeater.cc:503] Master 127.7.67.254:37863 was elected leader, sending a full tablet report...
I20260806 18:29:37.874429 8760 heartbeater.cc:503] Master 127.7.67.254:37863 was elected leader, sending a full tablet report...
I20260806 18:29:37.875604 8996 ts_tablet_manager.cc:1428] T 21f28a11b11e4c9b911a891731e1796e P c709f45ee1954c4da2719309fc47b9e5: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:29:37.876526 8994 ts_tablet_manager.cc:1428] T 21f28a11b11e4c9b911a891731e1796e P 8854301ae6dc48dfaaaa91f27ef72c3c: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260806 18:29:37.877295 9001 raft_consensus.cc:348] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8854301ae6dc48dfaaaa91f27ef72c3c" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 42345 } } peers { permanent_uuid: "c709f45ee1954c4da2719309fc47b9e5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 37123 } } peers { permanent_uuid: "d4bff7b1830c4b2fa96c9049e1f0fee9" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 41379 } }
I20260806 18:29:37.877558 9001 raft_consensus.cc:374] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:37.877620 9001 raft_consensus.cc:729] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4bff7b1830c4b2fa96c9049e1f0fee9, State: Initialized, Role: FOLLOWER
I20260806 18:29:37.878108 9001 consensus_queue.cc:260] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8854301ae6dc48dfaaaa91f27ef72c3c" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 42345 } } peers { permanent_uuid: "c709f45ee1954c4da2719309fc47b9e5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 37123 } } peers { permanent_uuid: "d4bff7b1830c4b2fa96c9049e1f0fee9" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 41379 } }
I20260806 18:29:37.879395 9001 ts_tablet_manager.cc:1428] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 18:29:37.879585 8557 heartbeater.cc:503] Master 127.7.67.254:37863 was elected leader, sending a full tablet report...
I20260806 18:29:37.929368 9006 raft_consensus.cc:482] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:29:37.929617 9006 raft_consensus.cc:504] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8854301ae6dc48dfaaaa91f27ef72c3c" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 42345 } } peers { permanent_uuid: "c709f45ee1954c4da2719309fc47b9e5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 37123 } } peers { permanent_uuid: "d4bff7b1830c4b2fa96c9049e1f0fee9" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 41379 } }
I20260806 18:29:37.930789 9006 leader_election.cc:288] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8854301ae6dc48dfaaaa91f27ef72c3c (127.7.67.195:42345), c709f45ee1954c4da2719309fc47b9e5 (127.7.67.194:37123)
I20260806 18:29:37.935470 8983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.931161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:43609 (local address 127.7.67.195:42345)
0806 18:29:37.931333 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:37.931338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:37.931521 (+ 183us) server_negotiation.cc:407] Connection header received
0806 18:29:37.931724 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:37.931728 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:37.931792 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:37.931879 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:37.932191 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.932917 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.934370 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.934974 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.935096 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:37.935148 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:37.935286 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 18:29:37.935352 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:29:37.935493 9008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.931075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:42345 (local address 127.7.67.193:43609)
0806 18:29:37.931416 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:37.931429 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:37.931558 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:37.931929 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:37.931932 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:37.931955 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:37.932095 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:37.932103 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.933030 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.933032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:37.934249 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:37.934261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.935081 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.935083 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:37.935158 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:37.935181 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:29:37.935257 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 18:29:37.935328 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":169,"threads_started":1}
I20260806 18:29:37.935640 8990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.931694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:38191 (local address 127.7.67.194:37123)
0806 18:29:37.931819 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:37.931822 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:37.932119 (+ 297us) server_negotiation.cc:407] Connection header received
0806 18:29:37.932248 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:37.932251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:37.932301 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:37.932379 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:37.932587 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.933207 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.934528 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.935135 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.935235 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:37.935279 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:37.935464 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 18:29:37.935529 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:29:37.935644 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.931539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:37123 (local address 127.7.67.193:38191)
0806 18:29:37.932018 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:37.932031 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:37.932143 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:37.932388 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:37.932390 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:37.932403 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:37.932495 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:37.932499 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.933306 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.933309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:37.934426 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:37.934434 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.935240 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.935243 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:37.935352 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:37.935373 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:29:37.935439 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 18:29:37.935495 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":13,"thread_start_us":207,"threads_started":1}
I20260806 18:29:37.936232 8694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21f28a11b11e4c9b911a891731e1796e" candidate_uuid: "d4bff7b1830c4b2fa96c9049e1f0fee9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c709f45ee1954c4da2719309fc47b9e5" is_pre_election: true
I20260806 18:29:37.936239 8897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21f28a11b11e4c9b911a891731e1796e" candidate_uuid: "d4bff7b1830c4b2fa96c9049e1f0fee9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8854301ae6dc48dfaaaa91f27ef72c3c" is_pre_election: true
I20260806 18:29:37.936504 8694 raft_consensus.cc:2454] T 21f28a11b11e4c9b911a891731e1796e P c709f45ee1954c4da2719309fc47b9e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4bff7b1830c4b2fa96c9049e1f0fee9 in term 0.
I20260806 18:29:37.936514 8897 raft_consensus.cc:2454] T 21f28a11b11e4c9b911a891731e1796e P 8854301ae6dc48dfaaaa91f27ef72c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4bff7b1830c4b2fa96c9049e1f0fee9 in term 0.
I20260806 18:29:37.936954 8388 leader_election.cc:302] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8854301ae6dc48dfaaaa91f27ef72c3c, d4bff7b1830c4b2fa96c9049e1f0fee9; no voters:
I20260806 18:29:37.937222 9006 raft_consensus.cc:2781] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:29:37.937306 9006 raft_consensus.cc:482] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:29:37.937350 9006 raft_consensus.cc:3037] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:37.938400 9006 raft_consensus.cc:504] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8854301ae6dc48dfaaaa91f27ef72c3c" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 42345 } } peers { permanent_uuid: "c709f45ee1954c4da2719309fc47b9e5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 37123 } } peers { permanent_uuid: "d4bff7b1830c4b2fa96c9049e1f0fee9" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 41379 } }
I20260806 18:29:37.938714 9006 leader_election.cc:288] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9 [CANDIDATE]: Term 1 election: Requested vote from peers 8854301ae6dc48dfaaaa91f27ef72c3c (127.7.67.195:42345), c709f45ee1954c4da2719309fc47b9e5 (127.7.67.194:37123)
I20260806 18:29:37.939090 8897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21f28a11b11e4c9b911a891731e1796e" candidate_uuid: "d4bff7b1830c4b2fa96c9049e1f0fee9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8854301ae6dc48dfaaaa91f27ef72c3c"
I20260806 18:29:37.939178 8694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21f28a11b11e4c9b911a891731e1796e" candidate_uuid: "d4bff7b1830c4b2fa96c9049e1f0fee9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c709f45ee1954c4da2719309fc47b9e5"
I20260806 18:29:37.939205 8897 raft_consensus.cc:3037] T 21f28a11b11e4c9b911a891731e1796e P 8854301ae6dc48dfaaaa91f27ef72c3c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:37.939301 8694 raft_consensus.cc:3037] T 21f28a11b11e4c9b911a891731e1796e P c709f45ee1954c4da2719309fc47b9e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:37.940236 8897 raft_consensus.cc:2454] T 21f28a11b11e4c9b911a891731e1796e P 8854301ae6dc48dfaaaa91f27ef72c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bff7b1830c4b2fa96c9049e1f0fee9 in term 1.
I20260806 18:29:37.940349 8694 raft_consensus.cc:2454] T 21f28a11b11e4c9b911a891731e1796e P c709f45ee1954c4da2719309fc47b9e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bff7b1830c4b2fa96c9049e1f0fee9 in term 1.
I20260806 18:29:37.940585 8388 leader_election.cc:302] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8854301ae6dc48dfaaaa91f27ef72c3c, d4bff7b1830c4b2fa96c9049e1f0fee9; no voters:
I20260806 18:29:37.940758 9006 raft_consensus.cc:2781] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:37.940956 9006 raft_consensus.cc:686] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9 [term 1 LEADER]: Becoming Leader. State: Replica: d4bff7b1830c4b2fa96c9049e1f0fee9, State: Running, Role: LEADER
I20260806 18:29:37.941272 9006 consensus_queue.cc:237] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8854301ae6dc48dfaaaa91f27ef72c3c" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 42345 } } peers { permanent_uuid: "c709f45ee1954c4da2719309fc47b9e5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 37123 } } peers { permanent_uuid: "d4bff7b1830c4b2fa96c9049e1f0fee9" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 41379 } }
I20260806 18:29:37.944537 8263 catalog_manager.cc:5495] T 21f28a11b11e4c9b911a891731e1796e P d4bff7b1830c4b2fa96c9049e1f0fee9 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4bff7b1830c4b2fa96c9049e1f0fee9 (127.7.67.193). New cstate: current_term: 1 leader_uuid: "d4bff7b1830c4b2fa96c9049e1f0fee9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8854301ae6dc48dfaaaa91f27ef72c3c" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 42345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c709f45ee1954c4da2719309fc47b9e5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 37123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4bff7b1830c4b2fa96c9049e1f0fee9" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 41379 } health_report { overall_health: HEALTHY } } }
Aug 06 18:29:37 dist-test-slave-5bn2 krb5kdc[8198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.193@KRBTEST.COM
I20260806 18:29:37.975989 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.967774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40462 (local address 127.7.67.193:41379)
0806 18:29:37.967899 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:37.967903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:37.968109 (+ 206us) server_negotiation.cc:407] Connection header received
0806 18:29:37.968210 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:37.968212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:37.968255 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:37.968323 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:37.968952 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.969460 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.970222 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.970419 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.970487 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:37.972874 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:37.972887 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:37.972905 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:37.972930 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:37.974356 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:37.974735 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:37.974740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:37.974741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:37.974818 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:37.975122 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:37.975124 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:37.975125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:37.975456 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:37.975631 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:37.975753 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 18:29:37.975857 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 18:29:37 dist-test-slave-5bn2 krb5kdc[8198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040977, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:29:37 dist-test-slave-5bn2 krb5kdc[8198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040977, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.67.193@KRBTEST.COM
I20260806 18:29:38.003968 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:37.996378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40466 (local address 127.7.67.193:41379)
0806 18:29:37.996504 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:37.996507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:37.996727 (+ 220us) server_negotiation.cc:407] Connection header received
0806 18:29:37.996847 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:37.996850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:37.996896 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:37.996961 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:37.997565 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.998089 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.998818 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:37.999095 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:37.999171 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:38.001361 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:38.001365 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:38.001367 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:38.001391 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:38.002651 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:38.003034 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:38.003036 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:38.003038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:38.003111 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:38.003295 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:38.003297 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:38.003298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:38.003447 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:38.003551 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:38.003741 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 18:29:38.003831 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 18:29:38 dist-test-slave-5bn2 krb5kdc[8198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:29:38 dist-test-slave-5bn2 krb5kdc[8198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.193@KRBTEST.COM
I20260806 18:29:38.032538 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:38.024422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40468 (local address 127.7.67.193:41379)
0806 18:29:38.024610 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:38.024615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:38.024709 (+ 94us) server_negotiation.cc:407] Connection header received
0806 18:29:38.024882 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:38.024885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:38.024933 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:38.025003 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:38.025751 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:38.026263 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:38.027150 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:38.027369 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:38.027458 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:38.029745 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:38.029748 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:38.029750 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:38.029776 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:38.030984 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:38.031383 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:38.031386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:38.031388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:38.031470 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:38.031716 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:38.031720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:38.031722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:38.031897 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:38.032037 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:38.032259 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 18:29:38.032372 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 18:29:38.033969 8427 tablet_service.cc:2908] Rejecting scan request for tablet 21f28a11b11e4c9b911a891731e1796e: Uninitialized: clean time has not yet been initialized
I20260806 18:29:38.036181 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 8367
W20260806 18:29:38.039286 8968 tablet.cc:2369] T 21f28a11b11e4c9b911a891731e1796e P 8854301ae6dc48dfaaaa91f27ef72c3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:29:38.046870 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 8566
I20260806 18:29:38.057147 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 8765
I20260806 18:29:38.067371 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 8215
2026-08-06T18:29:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2431 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:29:38 dist-test-slave-5bn2 krb5kdc[9035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:29:38 dist-test-slave-5bn2 krb5kdc[9035](info): set up 2 sockets
Aug 06 18:29:38 dist-test-slave-5bn2 krb5kdc[9035](info): commencing operation
krb5kdc: starting...
W20260806 18:29:39.186623 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260806 18:29:39.400960 7439 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.214s user 0.002s sys 0.000s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260806 18:29:39.508646 7439 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.108s user 0.002s sys 0.000s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260806 18:29:39.655740 7439 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.147s user 0.001s sys 0.001s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/test-user.keytab.
W20260806 18:29:40.244874 7439 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.589s user 0.001s sys 0.001s
Aug 06 18:29:40 dist-test-slave-5bn2 krb5kdc[9035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:29:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:29:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
W20260806 18:29:40.469125 7439 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.7.67.254: real 0.198s user 0.000s sys 0.003s
I20260806 18:29:40.469632 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:42143
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:38581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:42143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:40.584302 9059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:40.584597 9059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:40.584653 9059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:40.588758 9059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:29:40.588820 9059 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 18:29:40.588842 9059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:40.588864 9059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:40.588886 9059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:29:40.588914 9059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:29:40.594260 9059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:38581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:42143
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:42143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:40.595357 9059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:40.596377 9059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:40.604852 9066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:40.604926 9059 server_base.cc:1034] running on GCE node
W20260806 18:29:40.604787 9071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:40.604787 9067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:40.605707 9059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:40.606814 9059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:40.608029 9059 hybrid_clock.cc:648] HybridClock initialized: now 1786040980608006 us; error 32 us; skew 500 ppm
Aug 06 18:29:40 dist-test-slave-5bn2 krb5kdc[9035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040980, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:40.614012 9059 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:29:40.615000 9059 webserver.cc:460] Webserver started at http://127.7.67.254:39157/ using document root <none> and password file <none>
I20260806 18:29:40.615561 9059 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:40.615622 9059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:40.615832 9059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:40.617946 9059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "45ba319e405846dea53a73b8e222f17c"
format_stamp: "Formatted at 2026-08-06 18:29:40 on dist-test-slave-5bn2"
server_key: "5154e666212ff79d8bc88b849f16026f"
server_key_iv: "a49798ba818d8181a8c9981d270ccf4b"
server_key_version: "encryptionkey@0"
I20260806 18:29:40.618512 9059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "45ba319e405846dea53a73b8e222f17c"
format_stamp: "Formatted at 2026-08-06 18:29:40 on dist-test-slave-5bn2"
server_key: "5154e666212ff79d8bc88b849f16026f"
server_key_iv: "a49798ba818d8181a8c9981d270ccf4b"
server_key_version: "encryptionkey@0"
I20260806 18:29:40.622371 9059 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260806 18:29:40.624912 9080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:40.626180 9059 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:29:40.626292 9059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "45ba319e405846dea53a73b8e222f17c"
format_stamp: "Formatted at 2026-08-06 18:29:40 on dist-test-slave-5bn2"
server_key: "5154e666212ff79d8bc88b849f16026f"
server_key_iv: "a49798ba818d8181a8c9981d270ccf4b"
server_key_version: "encryptionkey@0"
I20260806 18:29:40.626391 9059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:40.650123 9059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:40.653100 9059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:40.653270 9059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:40.661458 9059 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:42143
I20260806 18:29:40.661521 9171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:42143 every 8 connection(s)
I20260806 18:29:40.662454 9059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:29:40.665273 9172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:40.666155 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 9059
I20260806 18:29:40.666261 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:29:40.666501 7439 external_mini_cluster.cc:1428] Setting key 7b7ecc4c0b05ddb7a1e2a1aeb53c2845
I20260806 18:29:40.671545 9172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c: Bootstrap starting.
I20260806 18:29:40.673838 9172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:40.674549 9172 log.cc:824] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:29:40 dist-test-slave-5bn2 krb5kdc[9035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:40.676676 9172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c: No bootstrap required, opened a new log
I20260806 18:29:40.680263 9172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ba319e405846dea53a73b8e222f17c" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 42143 } }
I20260806 18:29:40.680559 9172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:40.680647 9172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ba319e405846dea53a73b8e222f17c, State: Initialized, Role: FOLLOWER
I20260806 18:29:40.681149 9172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ba319e405846dea53a73b8e222f17c" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 42143 } }
I20260806 18:29:40.681305 9172 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:29:40.681409 9172 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:29:40.681468 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:40.667888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50092 (local address 127.7.67.254:42143)
0806 18:29:40.668307 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:40.668323 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:40.668360 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:29:40.669007 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:40.669024 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:40.669355 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:40.669644 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:40.670514 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:40.671543 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:40.672393 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:40.672874 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:40.673024 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:40.675515 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:40.675521 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:40.675535 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:40.675571 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:40.677972 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:40.678490 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:40.678496 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:40.678502 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:40.678667 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:40.679036 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:40.679059 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:40.679061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:40.679623 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:40.679832 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:40.680151 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 18:29:40.680460 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":125,"threads_started":1}
I20260806 18:29:40.681516 9172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:40.682497 9172 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ba319e405846dea53a73b8e222f17c" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 42143 } }
I20260806 18:29:40.682886 9172 leader_election.cc:302] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45ba319e405846dea53a73b8e222f17c; no voters:
I20260806 18:29:40.683264 9172 leader_election.cc:288] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:29:40.683367 9180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:40.683584 9180 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c [term 1 LEADER]: Becoming Leader. State: Replica: 45ba319e405846dea53a73b8e222f17c, State: Running, Role: LEADER
I20260806 18:29:40.683954 9180 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ba319e405846dea53a73b8e222f17c" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 42143 } }
I20260806 18:29:40.684518 9172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:29:40.685088 9180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45ba319e405846dea53a73b8e222f17c. Latest consensus state: current_term: 1 leader_uuid: "45ba319e405846dea53a73b8e222f17c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ba319e405846dea53a73b8e222f17c" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 42143 } } }
I20260806 18:29:40.685293 9180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c [sys.catalog]: This master's current role is: LEADER
I20260806 18:29:40.685592 9182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45ba319e405846dea53a73b8e222f17c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ba319e405846dea53a73b8e222f17c" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 42143 } } }
I20260806 18:29:40.685720 9182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c [sys.catalog]: This master's current role is: LEADER
I20260806 18:29:40.685819 9193 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:29:40.689200 9193 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:29:40.695007 9193 catalog_manager.cc:1347] Generated new cluster ID: 06eaaff592e9464798440fe874ddcc2a
I20260806 18:29:40.695071 9193 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:29:40.710139 9193 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:29:40.711129 9193 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:29:40.714609 9193 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c: Generated new TSK 0
I20260806 18:29:40.715332 9193 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
I20260806 18:29:40.751732 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:42143
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:38581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:40.896744 9213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:40.897009 9213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:40.897056 9213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:40.901070 9213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:40.901129 9213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:40.901175 9213 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:29:40.901225 9213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:29:40.906405 9213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:38581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.67.254:42143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:40.907455 9213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:40.908464 9213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:40.918035 9230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:40.918036 9225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:40.918037 9226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:40.918385 9213 server_base.cc:1034] running on GCE node
I20260806 18:29:40.918778 9213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:40.919431 9213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:40.920588 9213 hybrid_clock.cc:648] HybridClock initialized: now 1786040980920560 us; error 41 us; skew 500 ppm
Aug 06 18:29:40 dist-test-slave-5bn2 krb5kdc[9035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040980, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:40.923326 9213 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:29:40.924235 9213 webserver.cc:460] Webserver started at http://127.7.67.193:34639/ using document root <none> and password file <none>
I20260806 18:29:40.924800 9213 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:40.924871 9213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:40.925079 9213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:40.927227 9213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "6619187e8db74ec78270cef5f3e3d223"
format_stamp: "Formatted at 2026-08-06 18:29:40 on dist-test-slave-5bn2"
server_key: "d0fbc57df84d850ac4e744e0d07feba3"
server_key_iv: "5459ac85a372c35f54f1428d4b118b6a"
server_key_version: "encryptionkey@0"
I20260806 18:29:40.927798 9213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "6619187e8db74ec78270cef5f3e3d223"
format_stamp: "Formatted at 2026-08-06 18:29:40 on dist-test-slave-5bn2"
server_key: "d0fbc57df84d850ac4e744e0d07feba3"
server_key_iv: "5459ac85a372c35f54f1428d4b118b6a"
server_key_version: "encryptionkey@0"
I20260806 18:29:40.931579 9213 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:29:40.933938 9242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:40.935149 9213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260806 18:29:40.935263 9213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "6619187e8db74ec78270cef5f3e3d223"
format_stamp: "Formatted at 2026-08-06 18:29:40 on dist-test-slave-5bn2"
server_key: "d0fbc57df84d850ac4e744e0d07feba3"
server_key_iv: "5459ac85a372c35f54f1428d4b118b6a"
server_key_version: "encryptionkey@0"
I20260806 18:29:40.935362 9213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:40.946725 9213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:40.949467 9213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:40.949661 9213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:40.951184 9213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:40.951236 9213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:40.951292 9213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:40.951320 9213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:29:40 dist-test-slave-5bn2 krb5kdc[9035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040980, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:40.961671 9213 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:33527
I20260806 18:29:40.962021 9432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:33527 every 8 connection(s)
I20260806 18:29:40.962634 9213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:29:40.964643 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:40.951919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:53949 (local address 127.7.67.254:42143)
0806 18:29:40.952064 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:40.952069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:40.952924 (+ 855us) server_negotiation.cc:407] Connection header received
0806 18:29:40.953925 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:40.953929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:40.953991 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:40.954104 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:40.955455 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:40.955986 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:40.957229 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:40.957535 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:40.957639 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:40.961133 (+ 3494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:40.961137 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:40.961140 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:40.961168 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:40.962620 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:40.963194 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:40.963197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:40.963199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:40.963292 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:40.963695 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:40.963698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:40.963700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:40.963901 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:40.963984 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:40.964404 (+ 420us) server_negotiation.cc:299] Negotiation successful
0806 18:29:40.964513 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:29:40.965378 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:40.952302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:42143 (local address 127.7.67.193:53949)
0806 18:29:40.952747 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:40.952797 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:40.953687 (+ 890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:40.954269 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:40.954278 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:40.954729 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:40.955306 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:40.955323 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:40.956118 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:40.956122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:40.957104 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:40.957115 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:40.957688 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:40.957693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:40.957855 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:40.958378 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:40.958395 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:40.960964 (+ 2569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:40.962733 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:40.962739 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:40.962750 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:40.963071 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:40.963396 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:40.963399 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:40.963401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:40.963576 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:40.963983 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:40.963988 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:40.964210 (+ 222us) client_negotiation.cc:769] Sending connection context
0806 18:29:40.964354 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 18:29:40.964612 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":106,"threads_started":1}
I20260806 18:29:40.966493 9433 heartbeater.cc:347] Connected to a master server at 127.7.67.254:42143
I20260806 18:29:40.966707 9433 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:40.967267 9433 heartbeater.cc:511] Master 127.7.67.254:42143 requested a full tablet report, sending...
I20260806 18:29:40.968783 9103 ts_manager.cc:194] Registered new tserver with Master: 6619187e8db74ec78270cef5f3e3d223 (127.7.67.193:33527)
I20260806 18:29:40.969550 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 9213
I20260806 18:29:40.969672 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:29:40.969981 7439 external_mini_cluster.cc:1428] Setting key fad1ef57d267af20eecd6ecafa55c189
I20260806 18:29:40.970304 9103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:53949
I20260806 18:29:40.975864 9440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:40.968851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:42143 (local address 127.7.67.193:51091)
0806 18:29:40.969248 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:40.969260 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:40.969359 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:40.969584 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:40.969587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:40.969827 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:40.969951 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:40.969958 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:40.970935 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:40.970939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:40.971614 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:40.971625 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:40.972042 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:40.972046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:40.972133 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:40.972584 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:40.972598 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:40.972912 (+ 314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:40.974541 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:40.974544 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:40.974546 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:40.974741 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:40.975024 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:40.975027 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:40.975028 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:40.975110 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:40.975467 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:40.975469 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:40.975541 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 18:29:40.975615 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:29:40.975719 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":233,"threads_started":1}
I20260806 18:29:40.975935 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:40.968902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:51091 (local address 127.7.67.254:42143)
0806 18:29:40.969014 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:40.969018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:40.969334 (+ 316us) server_negotiation.cc:407] Connection header received
0806 18:29:40.969432 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:40.969435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:40.969477 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:40.969555 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:40.970087 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:40.970760 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:40.971729 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:40.971937 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:40.972022 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:40.973039 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:40.973042 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:40.973045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:40.973069 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:40.974434 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:40.974820 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:40.974824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:40.974825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:40.974940 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:40.975186 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:40.975189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:40.975191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:40.975386 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:40.975473 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:40.975687 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 18:29:40.975797 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
WARNING: no policy specified for kudu/127.7.67.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
I20260806 18:29:40.995590 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:42143
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:38581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:41.108922 9445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:41.109205 9445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:41.109249 9445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:41.113339 9445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:41.113406 9445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:41.113445 9445 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:29:41.113476 9445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:29:41.118717 9445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:38581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.67.254:42143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:41.119752 9445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:41.120731 9445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:41.130200 9451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:41.130174 9458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:41.130213 9452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:41.130406 9445 server_base.cc:1034] running on GCE node
I20260806 18:29:41.130939 9445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:41.131546 9445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:41.132716 9445 hybrid_clock.cc:648] HybridClock initialized: now 1786040981132699 us; error 33 us; skew 500 ppm
Aug 06 18:29:41 dist-test-slave-5bn2 krb5kdc[9035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040981, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:41.135582 9445 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:29:41.136427 9445 webserver.cc:460] Webserver started at http://127.7.67.194:42881/ using document root <none> and password file <none>
I20260806 18:29:41.136974 9445 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:41.137041 9445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:41.137231 9445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:41.139415 9445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "50cf4493c6aa44cfa0b16c5c96ead72c"
format_stamp: "Formatted at 2026-08-06 18:29:41 on dist-test-slave-5bn2"
server_key: "098c3c404f8325dd2d71b2e2fc167562"
server_key_iv: "22ca03072e39a24da77680c923f2fd51"
server_key_version: "encryptionkey@0"
I20260806 18:29:41.139961 9445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "50cf4493c6aa44cfa0b16c5c96ead72c"
format_stamp: "Formatted at 2026-08-06 18:29:41 on dist-test-slave-5bn2"
server_key: "098c3c404f8325dd2d71b2e2fc167562"
server_key_iv: "22ca03072e39a24da77680c923f2fd51"
server_key_version: "encryptionkey@0"
I20260806 18:29:41.143616 9445 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:29:41.145972 9471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:41.147161 9445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:29:41.147271 9445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "50cf4493c6aa44cfa0b16c5c96ead72c"
format_stamp: "Formatted at 2026-08-06 18:29:41 on dist-test-slave-5bn2"
server_key: "098c3c404f8325dd2d71b2e2fc167562"
server_key_iv: "22ca03072e39a24da77680c923f2fd51"
server_key_version: "encryptionkey@0"
I20260806 18:29:41.147368 9445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:41.166256 9445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:41.169481 9445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:41.169669 9445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:41.171241 9445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:41.171294 9445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:41.171351 9445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:41.171379 9445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:29:41 dist-test-slave-5bn2 krb5kdc[9035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040981, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:41.181273 9445 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:42497
I20260806 18:29:41.181313 9654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:42497 every 8 connection(s)
I20260806 18:29:41.182245 9445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:29:41.185557 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.171977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:55237 (local address 127.7.67.254:42143)
0806 18:29:41.172128 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:41.172132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:41.173046 (+ 914us) server_negotiation.cc:407] Connection header received
0806 18:29:41.174185 (+ 1139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:41.174190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:41.174255 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:41.174356 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:41.175928 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.176745 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.178355 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.178701 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.178820 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.181803 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:41.181808 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:41.181810 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:41.181842 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:41.183426 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:41.183964 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:41.183968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:41.183969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:41.184051 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:41.184369 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:41.184372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:41.184374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:41.184575 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:41.184661 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:41.185231 (+ 570us) server_negotiation.cc:299] Negotiation successful
0806 18:29:41.185374 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:29:41.186395 9482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.172401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:42143 (local address 127.7.67.194:55237)
0806 18:29:41.172865 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:41.172918 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:41.173930 (+ 1012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:41.174554 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:41.174563 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:41.175026 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:41.175752 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.175775 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.176870 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.176874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.178231 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.178242 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.178872 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.178877 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.179045 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.179740 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:41.179763 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:41.181585 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:41.183541 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:41.183547 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:41.183561 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:41.183867 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:41.184135 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:41.184138 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:41.184140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:41.184278 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:41.184677 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:41.184684 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:41.184965 (+ 281us) client_negotiation.cc:769] Sending connection context
0806 18:29:41.185162 (+ 197us) client_negotiation.cc:240] Negotiation successful
0806 18:29:41.185501 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":108,"threads_started":1}
I20260806 18:29:41.187603 9655 heartbeater.cc:347] Connected to a master server at 127.7.67.254:42143
I20260806 18:29:41.187757 9655 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:41.188253 9655 heartbeater.cc:511] Master 127.7.67.254:42143 requested a full tablet report, sending...
I20260806 18:29:41.189266 9103 ts_manager.cc:194] Registered new tserver with Master: 50cf4493c6aa44cfa0b16c5c96ead72c (127.7.67.194:42497)
I20260806 18:29:41.189953 9103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:55237
I20260806 18:29:41.192044 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 9445
I20260806 18:29:41.192166 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:29:41.192409 7439 external_mini_cluster.cc:1428] Setting key 23a6166a65a90ff7075b98c8d63c5f48
I20260806 18:29:41.196720 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.189798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:58541 (local address 127.7.67.254:42143)
0806 18:29:41.189902 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:41.189905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:41.189977 (+ 72us) server_negotiation.cc:407] Connection header received
0806 18:29:41.190093 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:41.190095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:41.190135 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:41.190214 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:41.190653 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.191405 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.192167 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.192421 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.192510 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.193868 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:41.193871 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:41.193873 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:41.193893 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:41.195139 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:41.195518 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:41.195522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:41.195524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:41.195601 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:41.195894 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:41.195898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:41.195900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:41.196147 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:41.196231 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:41.196497 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 18:29:41.196605 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:29:41.196758 9661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.189657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:42143 (local address 127.7.67.194:58541)
0806 18:29:41.189914 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:41.189926 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:41.190009 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:41.190236 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:41.190239 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:41.190437 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:41.190545 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.190551 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.191522 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.191525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.192058 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.192066 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.192534 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.192538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.192627 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.193149 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:41.193162 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:41.193559 (+ 397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:41.195236 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:41.195239 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:41.195241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:41.195431 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:41.195681 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:41.195684 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:41.195686 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:41.195776 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:41.196243 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:41.196246 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:41.196345 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 18:29:41.196444 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 18:29:41.196587 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for kudu/127.7.67.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
I20260806 18:29:41.219759 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:42143
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:38581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:41.331885 9666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:41.332149 9666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:41.332191 9666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:41.336155 9666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:41.336215 9666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:41.336252 9666 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:29:41.336289 9666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:29:41.341554 9666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:38581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.67.254:42143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:41.342543 9666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:41.343569 9666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:41.353139 9680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:41.353160 9683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:41.353152 9681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:41.353240 9666 server_base.cc:1034] running on GCE node
I20260806 18:29:41.353785 9666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:41.354430 9666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:41.355588 9666 hybrid_clock.cc:648] HybridClock initialized: now 1786040981355567 us; error 34 us; skew 500 ppm
Aug 06 18:29:41 dist-test-slave-5bn2 krb5kdc[9035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040981, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:41.358229 9666 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:29:41.359114 9666 webserver.cc:460] Webserver started at http://127.7.67.195:35661/ using document root <none> and password file <none>
I20260806 18:29:41.359645 9666 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:41.359709 9666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:41.359907 9666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:41.361944 9666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "229f009cdba345778e38af50ee15bb00"
format_stamp: "Formatted at 2026-08-06 18:29:41 on dist-test-slave-5bn2"
server_key: "393384b99ac5f5ba43a865bd27632e2c"
server_key_iv: "0eba8f72f1401d1436e3f0f4872aedce"
server_key_version: "encryptionkey@0"
I20260806 18:29:41.362499 9666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "229f009cdba345778e38af50ee15bb00"
format_stamp: "Formatted at 2026-08-06 18:29:41 on dist-test-slave-5bn2"
server_key: "393384b99ac5f5ba43a865bd27632e2c"
server_key_iv: "0eba8f72f1401d1436e3f0f4872aedce"
server_key_version: "encryptionkey@0"
I20260806 18:29:41.366276 9666 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:29:41.368788 9694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:41.369956 9666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:29:41.370081 9666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "229f009cdba345778e38af50ee15bb00"
format_stamp: "Formatted at 2026-08-06 18:29:41 on dist-test-slave-5bn2"
server_key: "393384b99ac5f5ba43a865bd27632e2c"
server_key_iv: "0eba8f72f1401d1436e3f0f4872aedce"
server_key_version: "encryptionkey@0"
I20260806 18:29:41.370184 9666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:41.388197 9666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:41.391109 9666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:41.391268 9666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:41.392726 9666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:41.392779 9666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:41.392824 9666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:41.392853 9666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:29:41 dist-test-slave-5bn2 krb5kdc[9035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040981, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:41.403472 9666 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:46289
I20260806 18:29:41.403514 9889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:46289 every 8 connection(s)
I20260806 18:29:41.404441 9666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:29:41.405895 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 9666
I20260806 18:29:41.406013 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:29:41.406270 7439 external_mini_cluster.cc:1428] Setting key 1319ae93b0efdf9069824f970d490406
I20260806 18:29:41.406991 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.393578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:38851 (local address 127.7.67.254:42143)
0806 18:29:41.393696 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:41.393700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:41.394821 (+ 1121us) server_negotiation.cc:407] Connection header received
0806 18:29:41.395897 (+ 1076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:41.395901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:41.395966 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:41.396024 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:41.397753 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.398569 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.400030 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.400362 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.400477 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.403356 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:41.403360 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:41.403364 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:41.403396 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:41.404811 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:41.405411 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:41.405415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:41.405417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:41.405507 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:41.405834 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:41.405838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:41.405839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:41.406056 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:41.406148 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:41.406678 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 18:29:41.406805 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:29:41.407850 9707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.394152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:42143 (local address 127.7.67.195:38851)
0806 18:29:41.394672 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:41.394717 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:41.395648 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:41.396417 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:41.396426 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:41.396921 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:41.397568 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.397590 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.398710 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.398714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.399909 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.399920 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.400529 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.400533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.400734 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.401304 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:41.401324 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:41.403188 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:41.404943 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:41.404948 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:41.404962 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:41.405305 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:41.405589 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:41.405592 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:41.405595 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:41.405736 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:41.406149 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:41.406156 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:41.406412 (+ 256us) client_negotiation.cc:769] Sending connection context
0806 18:29:41.406591 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 18:29:41.406933 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":166,"threads_started":1}
I20260806 18:29:41.409080 9890 heartbeater.cc:347] Connected to a master server at 127.7.67.254:42143
I20260806 18:29:41.409224 9890 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:41.409739 9890 heartbeater.cc:511] Master 127.7.67.254:42143 requested a full tablet report, sending...
I20260806 18:29:41.410763 9103 ts_manager.cc:194] Registered new tserver with Master: 229f009cdba345778e38af50ee15bb00 (127.7.67.195:46289)
I20260806 18:29:41.411475 9103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:38851
I20260806 18:29:41.417258 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.411152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:45383 (local address 127.7.67.254:42143)
0806 18:29:41.411268 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:41.411272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:41.411437 (+ 165us) server_negotiation.cc:407] Connection header received
0806 18:29:41.411524 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:41.411528 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:41.411571 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:41.411647 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:41.412074 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.412795 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.413544 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.413749 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.413825 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.414764 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:41.414767 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:41.414769 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:41.414789 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:41.416079 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:41.416413 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:41.416416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:41.416417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:41.416482 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:41.416673 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:41.416675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:41.416676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:41.416822 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:41.416891 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:41.417063 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 18:29:41.417148 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:29:41.417261 9899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.411048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:42143 (local address 127.7.67.195:45383)
0806 18:29:41.411368 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:41.411380 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:41.411462 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:41.411663 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:41.411665 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:41.411874 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:41.411979 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.411986 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.412901 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.412904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.413445 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.413452 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.413833 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.413835 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.413906 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.414333 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:41.414344 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:41.414651 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:41.416160 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:41.416162 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:41.416164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:41.416342 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:41.416541 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:41.416542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:41.416543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:41.416616 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:41.416884 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:41.416886 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:41.416957 (+ 71us) client_negotiation.cc:769] Sending connection context
0806 18:29:41.417026 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 18:29:41.417129 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":150,"threads_started":1}
I20260806 18:29:41.420259 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:29:41 dist-test-slave-5bn2 krb5kdc[9035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040981, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:29:41 dist-test-slave-5bn2 krb5kdc[9035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040981, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:41.446412 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.439036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50102 (local address 127.7.67.254:42143)
0806 18:29:41.439179 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:41.439183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:41.439195 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:29:41.439239 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:41.439241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:41.439281 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:41.439347 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:41.440115 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.440625 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.441360 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.441554 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.441618 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.443815 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:41.443819 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:41.443821 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:41.443847 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:41.445135 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:41.445459 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:41.445461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:41.445463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:41.445538 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:41.445715 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:41.445717 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:41.445718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:41.445885 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:41.445982 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:41.446149 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 18:29:41.446245 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:29:41.449330 9103 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:29:41.452589 9103 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:29:41.467512 9917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.461355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:46289 (local address 127.0.0.1:38042)
0806 18:29:41.463017 (+ 1662us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:41.463030 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:41.463119 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:41.463598 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:41.463601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:41.463620 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:41.463834 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.463843 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.465099 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.465102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.466049 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.466060 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.467143 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.467146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.467230 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.467251 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:29:41.467308 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 18:29:41.467360 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1599,"mutex_wait_us":5,"thread_start_us":81,"threads_started":1}
I20260806 18:29:41.467711 9918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.461189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39256 (local address 127.7.67.193:33527)
0806 18:29:41.462421 (+ 1232us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:41.462437 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:41.462460 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:29:41.462526 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:41.462530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:41.462673 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:41.462804 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:41.463141 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.464155 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.465944 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.466760 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.466897 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.467060 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:41.467449 (+ 389us) server_negotiation.cc:299] Negotiation successful
0806 18:29:41.467552 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1125,"thread_start_us":109,"threads_started":1}
I20260806 18:29:41.468107 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.461841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38042 (local address 127.7.67.195:46289)
0806 18:29:41.463169 (+ 1328us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:41.463182 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:41.463204 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:29:41.463275 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:41.463280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:41.463453 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:41.463597 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:41.463960 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.464957 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.466177 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.466786 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.466907 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.467080 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:41.467646 (+ 566us) server_negotiation.cc:299] Negotiation successful
0806 18:29:41.467964 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1216,"thread_start_us":122,"threads_started":1}
I20260806 18:29:41.468218 9922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.461864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48584 (local address 127.7.67.194:42497)
0806 18:29:41.462334 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:41.462347 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:41.462372 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:29:41.462444 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:41.462448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:41.462614 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:41.462755 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:41.463153 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.464093 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.466207 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.467106 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.467207 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.467356 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:41.467966 (+ 610us) server_negotiation.cc:299] Negotiation successful
0806 18:29:41.468079 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":124,"threads_started":1}
I20260806 18:29:41.467671 9919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.461076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:42497 (local address 127.0.0.1:48584)
0806 18:29:41.461942 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:41.462009 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:41.462142 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:41.462766 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:41.462773 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:41.462797 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:41.463014 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.463021 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.464191 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.464194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.465836 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.465843 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.467223 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.467226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.467326 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.467354 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 18:29:41.467422 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 18:29:41.467510 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"mutex_wait_us":141,"thread_start_us":93,"threads_started":1}
I20260806 18:29:41.467512 9916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.461012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:33527 (local address 127.0.0.1:39256)
0806 18:29:41.461714 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:41.462153 (+ 439us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:41.462228 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:41.462800 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:41.462803 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:41.462823 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:41.463001 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.463010 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.464267 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.464270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.465831 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.465843 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.466907 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.466911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.467034 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.467090 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 18:29:41.467201 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 18:29:41.467330 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"mutex_wait_us":120,"thread_start_us":102,"threads_started":1}
I20260806 18:29:41.470832 9778 tablet_service.cc:1467] Processing CreateTablet for tablet b65ba7f2c5bf4c4c8abae07a55a676dd (DEFAULT_TABLE table=test-table [id=f0caf030d6d4434a849270061f7c12ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:41.470835 9544 tablet_service.cc:1467] Processing CreateTablet for tablet b65ba7f2c5bf4c4c8abae07a55a676dd (DEFAULT_TABLE table=test-table [id=f0caf030d6d4434a849270061f7c12ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:41.471908 9544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b65ba7f2c5bf4c4c8abae07a55a676dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:41.471935 9778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b65ba7f2c5bf4c4c8abae07a55a676dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:41.479125 9923 tablet_bootstrap.cc:492] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00: Bootstrap starting.
I20260806 18:29:41.479099 9328 tablet_service.cc:1467] Processing CreateTablet for tablet b65ba7f2c5bf4c4c8abae07a55a676dd (DEFAULT_TABLE table=test-table [id=f0caf030d6d4434a849270061f7c12ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:41.480283 9328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b65ba7f2c5bf4c4c8abae07a55a676dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:41.480751 9924 tablet_bootstrap.cc:492] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c: Bootstrap starting.
I20260806 18:29:41.481693 9923 tablet_bootstrap.cc:654] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:41.482432 9923 log.cc:824] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00: Log is configured to *not* fsync() on all Append() calls
I20260806 18:29:41.483045 9924 tablet_bootstrap.cc:654] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:41.483896 9924 log.cc:824] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c: Log is configured to *not* fsync() on all Append() calls
I20260806 18:29:41.484411 9923 tablet_bootstrap.cc:492] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00: No bootstrap required, opened a new log
I20260806 18:29:41.484648 9923 ts_tablet_manager.cc:1397] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:29:41.486136 9924 tablet_bootstrap.cc:492] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c: No bootstrap required, opened a new log
I20260806 18:29:41.486410 9924 ts_tablet_manager.cc:1397] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260806 18:29:41.488312 9928 tablet_bootstrap.cc:492] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223: Bootstrap starting.
I20260806 18:29:41.488147 9923 raft_consensus.cc:348] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.488418 9923 raft_consensus.cc:374] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:41.488472 9923 raft_consensus.cc:729] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229f009cdba345778e38af50ee15bb00, State: Initialized, Role: FOLLOWER
I20260806 18:29:41.488971 9923 consensus_queue.cc:260] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.490028 9890 heartbeater.cc:503] Master 127.7.67.254:42143 was elected leader, sending a full tablet report...
I20260806 18:29:41.490334 9923 ts_tablet_manager.cc:1428] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:29:41.490744 9928 tablet_bootstrap.cc:654] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:41.490651 9924 raft_consensus.cc:348] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.490927 9924 raft_consensus.cc:374] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:41.490989 9924 raft_consensus.cc:729] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50cf4493c6aa44cfa0b16c5c96ead72c, State: Initialized, Role: FOLLOWER
I20260806 18:29:41.491426 9924 consensus_queue.cc:260] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.491577 9928 log.cc:824] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223: Log is configured to *not* fsync() on all Append() calls
I20260806 18:29:41.492369 9924 ts_tablet_manager.cc:1428] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:29:41.492710 9655 heartbeater.cc:503] Master 127.7.67.254:42143 was elected leader, sending a full tablet report...
I20260806 18:29:41.494006 9928 tablet_bootstrap.cc:492] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223: No bootstrap required, opened a new log
I20260806 18:29:41.494239 9928 ts_tablet_manager.cc:1397] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:29:41.497656 9928 raft_consensus.cc:348] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.497879 9928 raft_consensus.cc:374] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:41.497928 9928 raft_consensus.cc:729] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6619187e8db74ec78270cef5f3e3d223, State: Initialized, Role: FOLLOWER
I20260806 18:29:41.498344 9928 consensus_queue.cc:260] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.499007 9433 heartbeater.cc:503] Master 127.7.67.254:42143 was elected leader, sending a full tablet report...
I20260806 18:29:41.499337 9928 ts_tablet_manager.cc:1428] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:29:41.607009 9935 raft_consensus.cc:482] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:29:41.607250 9935 raft_consensus.cc:504] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.608453 9935 leader_election.cc:288] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50cf4493c6aa44cfa0b16c5c96ead72c (127.7.67.194:42497), 229f009cdba345778e38af50ee15bb00 (127.7.67.195:46289)
I20260806 18:29:41.614095 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.608892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:42139 (local address 127.7.67.195:46289)
0806 18:29:41.609030 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:41.609034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:41.609286 (+ 252us) server_negotiation.cc:407] Connection header received
0806 18:29:41.609451 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:41.609455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:41.609514 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:41.609613 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:41.609992 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.610871 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.612798 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.613435 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.613573 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.613628 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:41.613889 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 18:29:41.613970 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 18:29:41.614122 9939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.608731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:46289 (local address 127.7.67.193:42139)
0806 18:29:41.609186 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:41.609203 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:41.609342 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:41.609632 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:41.609635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:41.609654 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:41.609879 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.609887 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.611026 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.611030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.612636 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.612651 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.613600 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.613604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.613739 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.613768 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:29:41.613845 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:29:41.613926 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":218,"spinlock_wait_cycles":768,"thread_start_us":156,"threads_started":1}
I20260806 18:29:41.614233 9922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.608893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:36105 (local address 127.7.67.194:42497)
0806 18:29:41.609029 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:41.609034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:41.609410 (+ 376us) server_negotiation.cc:407] Connection header received
0806 18:29:41.609513 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:41.609516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:41.609562 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:41.609629 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:41.609991 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.610867 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.612798 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.613457 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.613573 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.613628 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:41.614022 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 18:29:41.614107 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:29:41.614805 9936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.608732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:42497 (local address 127.7.67.193:36105)
0806 18:29:41.609327 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:41.609338 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:41.609420 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:41.609649 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:41.609652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:41.609664 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:41.609881 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.609887 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.611026 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.611030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.612636 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.612651 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.613701 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.613705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.613866 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.613898 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 18:29:41.614580 (+ 682us) client_negotiation.cc:240] Negotiation successful
0806 18:29:41.614660 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":257,"thread_start_us":132,"threads_started":1}
I20260806 18:29:41.614912 9813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b65ba7f2c5bf4c4c8abae07a55a676dd" candidate_uuid: "6619187e8db74ec78270cef5f3e3d223" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229f009cdba345778e38af50ee15bb00" is_pre_election: true
I20260806 18:29:41.615285 9813 raft_consensus.cc:2454] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6619187e8db74ec78270cef5f3e3d223 in term 0.
I20260806 18:29:41.615483 9575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b65ba7f2c5bf4c4c8abae07a55a676dd" candidate_uuid: "6619187e8db74ec78270cef5f3e3d223" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" is_pre_election: true
I20260806 18:29:41.615757 9575 raft_consensus.cc:2454] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6619187e8db74ec78270cef5f3e3d223 in term 0.
I20260806 18:29:41.615823 9245 leader_election.cc:302] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 229f009cdba345778e38af50ee15bb00, 6619187e8db74ec78270cef5f3e3d223; no voters:
I20260806 18:29:41.616151 9935 raft_consensus.cc:2781] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:29:41.616240 9935 raft_consensus.cc:482] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:29:41.616272 9935 raft_consensus.cc:3037] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:41.617352 9935 raft_consensus.cc:504] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.617704 9935 leader_election.cc:288] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [CANDIDATE]: Term 1 election: Requested vote from peers 50cf4493c6aa44cfa0b16c5c96ead72c (127.7.67.194:42497), 229f009cdba345778e38af50ee15bb00 (127.7.67.195:46289)
I20260806 18:29:41.618106 9575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b65ba7f2c5bf4c4c8abae07a55a676dd" candidate_uuid: "6619187e8db74ec78270cef5f3e3d223" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c"
I20260806 18:29:41.618100 9813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b65ba7f2c5bf4c4c8abae07a55a676dd" candidate_uuid: "6619187e8db74ec78270cef5f3e3d223" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229f009cdba345778e38af50ee15bb00"
I20260806 18:29:41.618216 9813 raft_consensus.cc:3037] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:41.618227 9575 raft_consensus.cc:3037] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:41.619347 9575 raft_consensus.cc:2454] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6619187e8db74ec78270cef5f3e3d223 in term 1.
I20260806 18:29:41.619501 9813 raft_consensus.cc:2454] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6619187e8db74ec78270cef5f3e3d223 in term 1.
I20260806 18:29:41.619679 9243 leader_election.cc:302] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50cf4493c6aa44cfa0b16c5c96ead72c, 6619187e8db74ec78270cef5f3e3d223; no voters:
I20260806 18:29:41.619872 9935 raft_consensus.cc:2781] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:41.620101 9935 raft_consensus.cc:686] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [term 1 LEADER]: Becoming Leader. State: Replica: 6619187e8db74ec78270cef5f3e3d223, State: Running, Role: LEADER
I20260806 18:29:41.620476 9935 consensus_queue.cc:237] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.623858 9107 catalog_manager.cc:5495] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6619187e8db74ec78270cef5f3e3d223 (127.7.67.193). New cstate: current_term: 1 leader_uuid: "6619187e8db74ec78270cef5f3e3d223" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } health_report { overall_health: UNKNOWN } } }
W20260806 18:29:41.655823 9891 tablet.cc:2369] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:29:41.667428 9918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.663421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39260 (local address 127.7.67.193:33527)
0806 18:29:41.663562 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:41.663565 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:41.663582 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:29:41.663805 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:41.663808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:41.663870 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:41.663963 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:29:41.664226 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.664725 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.665912 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.666137 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.666203 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.666556 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:29:41.667047 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:29:41.667134 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:41.667232 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 18:29:41.667288 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:29:41.673684 9813 raft_consensus.cc:1261] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00 [term 1 FOLLOWER]: Refusing update from remote peer 6619187e8db74ec78270cef5f3e3d223: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:29:41.673697 9575 raft_consensus.cc:1261] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c [term 1 FOLLOWER]: Refusing update from remote peer 6619187e8db74ec78270cef5f3e3d223: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:29:41.674439 9941 consensus_queue.cc:1035] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [LEADER]: Connected to new peer: Peer: permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:29:41.674652 9935 consensus_queue.cc:1035] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:29:41.682300 9945 mvcc.cc:205] Tried to move back new op lower bound from 7315623860928212992 to 7315623860720422912. Current Snapshot: MvccSnapshot[applied={T|T < 7315623860928212992}]
I20260806 18:29:41.696355 9103 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:29:41.696537 9103 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:29:41.699182 9103 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c: Sending DeleteTablet for 3 replicas of tablet b65ba7f2c5bf4c4c8abae07a55a676dd
I20260806 18:29:41.699973 9328 tablet_service.cc:1514] Processing DeleteTablet for tablet b65ba7f2c5bf4c4c8abae07a55a676dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:41 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:39256
I20260806 18:29:41.700060 9544 tablet_service.cc:1514] Processing DeleteTablet for tablet b65ba7f2c5bf4c4c8abae07a55a676dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:41 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:48584
I20260806 18:29:41.700071 9778 tablet_service.cc:1514] Processing DeleteTablet for tablet b65ba7f2c5bf4c4c8abae07a55a676dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:41 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:38042
I20260806 18:29:41.700508 9960 tablet_replica.cc:330] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223: stopping tablet replica
I20260806 18:29:41.700812 9960 raft_consensus.cc:2229] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:29:41.700953 9959 tablet_replica.cc:330] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c: stopping tablet replica
I20260806 18:29:41.701221 9960 raft_consensus.cc:2258] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:29:41.701267 9959 raft_consensus.cc:2229] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:29:41.701213 9103 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:29:41.701666 9103 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:29:41.701839 9959 raft_consensus.cc:2258] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:29:41.703001 9960 ts_tablet_manager.cc:1905] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:29:41.703251 9959 ts_tablet_manager.cc:1905] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:29:41.706045 9960 ts_tablet_manager.cc:1918] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:29:41.706182 9959 ts_tablet_manager.cc:1918] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:29:41.706287 9959 log.cc:1196] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/b65ba7f2c5bf4c4c8abae07a55a676dd
I20260806 18:29:41.706231 9960 log.cc:1196] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-0/wal/wals/b65ba7f2c5bf4c4c8abae07a55a676dd
I20260806 18:29:41.706601 9959 ts_tablet_manager.cc:1939] T b65ba7f2c5bf4c4c8abae07a55a676dd P 50cf4493c6aa44cfa0b16c5c96ead72c: Deleting consensus metadata
I20260806 18:29:41.706806 9960 ts_tablet_manager.cc:1939] T b65ba7f2c5bf4c4c8abae07a55a676dd P 6619187e8db74ec78270cef5f3e3d223: Deleting consensus metadata
I20260806 18:29:41.707960 9544 tablet_service.cc:1467] Processing CreateTablet for tablet 9b61db7a1d4947d3aa2c92e610a2f007 (DEFAULT_TABLE table=test-table [id=98e1cca7b43e412aa493915ff8e42aff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:41.708153 9778 tablet_service.cc:1467] Processing CreateTablet for tablet 9b61db7a1d4947d3aa2c92e610a2f007 (DEFAULT_TABLE table=test-table [id=98e1cca7b43e412aa493915ff8e42aff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:41.708220 9544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b61db7a1d4947d3aa2c92e610a2f007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:41.708403 9778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b61db7a1d4947d3aa2c92e610a2f007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:41.710381 9924 tablet_bootstrap.cc:492] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c: Bootstrap starting.
I20260806 18:29:41.710562 9081 catalog_manager.cc:4841] TS 50cf4493c6aa44cfa0b16c5c96ead72c (127.7.67.194:42497): tablet b65ba7f2c5bf4c4c8abae07a55a676dd (table test-table [id=f0caf030d6d4434a849270061f7c12ba]) successfully deleted
I20260806 18:29:41.711053 9082 catalog_manager.cc:4841] TS 6619187e8db74ec78270cef5f3e3d223 (127.7.67.193:33527): tablet b65ba7f2c5bf4c4c8abae07a55a676dd (table test-table [id=f0caf030d6d4434a849270061f7c12ba]) successfully deleted
I20260806 18:29:41.711372 9924 tablet_bootstrap.cc:654] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:41.711555 9962 tablet_replica.cc:330] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00: stopping tablet replica
I20260806 18:29:41.711891 9962 raft_consensus.cc:2229] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:29:41.712205 9962 raft_consensus.cc:2258] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:29:41.712360 9923 tablet_bootstrap.cc:492] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00: Bootstrap starting.
I20260806 18:29:41.712908 9328 tablet_service.cc:1467] Processing CreateTablet for tablet 9b61db7a1d4947d3aa2c92e610a2f007 (DEFAULT_TABLE table=test-table [id=98e1cca7b43e412aa493915ff8e42aff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:41.713164 9328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b61db7a1d4947d3aa2c92e610a2f007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:41.713356 9923 tablet_bootstrap.cc:654] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:41.713449 9924 tablet_bootstrap.cc:492] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c: No bootstrap required, opened a new log
I20260806 18:29:41.713527 9924 ts_tablet_manager.cc:1397] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260806 18:29:41.713992 9924 raft_consensus.cc:348] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.714108 9924 raft_consensus.cc:374] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:41.714143 9924 raft_consensus.cc:729] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50cf4493c6aa44cfa0b16c5c96ead72c, State: Initialized, Role: FOLLOWER
I20260806 18:29:41.714267 9924 consensus_queue.cc:260] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.714480 9923 tablet_bootstrap.cc:492] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00: No bootstrap required, opened a new log
I20260806 18:29:41.714555 9923 ts_tablet_manager.cc:1397] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:29:41.714780 9924 ts_tablet_manager.cc:1428] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260806 18:29:41.714999 9923 raft_consensus.cc:348] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.715111 9923 raft_consensus.cc:374] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:41.715138 9923 raft_consensus.cc:729] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229f009cdba345778e38af50ee15bb00, State: Initialized, Role: FOLLOWER
I20260806 18:29:41.715257 9923 consensus_queue.cc:260] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.715517 9923 ts_tablet_manager.cc:1428] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:29:41.716634 9962 ts_tablet_manager.cc:1905] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:29:41.718103 9928 tablet_bootstrap.cc:492] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223: Bootstrap starting.
I20260806 18:29:41.719144 9928 tablet_bootstrap.cc:654] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:41.719307 9962 ts_tablet_manager.cc:1918] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:29:41.719406 9962 log.cc:1196] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/b65ba7f2c5bf4c4c8abae07a55a676dd
I20260806 18:29:41.719720 9962 ts_tablet_manager.cc:1939] T b65ba7f2c5bf4c4c8abae07a55a676dd P 229f009cdba345778e38af50ee15bb00: Deleting consensus metadata
I20260806 18:29:41.720796 9928 tablet_bootstrap.cc:492] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223: No bootstrap required, opened a new log
I20260806 18:29:41.720890 9928 ts_tablet_manager.cc:1397] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:29:41.721076 9085 catalog_manager.cc:4841] TS 229f009cdba345778e38af50ee15bb00 (127.7.67.195:46289): tablet b65ba7f2c5bf4c4c8abae07a55a676dd (table test-table [id=f0caf030d6d4434a849270061f7c12ba]) successfully deleted
I20260806 18:29:41.721374 9928 raft_consensus.cc:348] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.721486 9928 raft_consensus.cc:374] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:41.721509 9928 raft_consensus.cc:729] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6619187e8db74ec78270cef5f3e3d223, State: Initialized, Role: FOLLOWER
I20260806 18:29:41.721638 9928 consensus_queue.cc:260] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.721889 9928 ts_tablet_manager.cc:1428] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:29:41.755306 9933 raft_consensus.cc:482] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:29:41.755466 9933 raft_consensus.cc:504] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.756735 9933 leader_election.cc:288] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6619187e8db74ec78270cef5f3e3d223 (127.7.67.193:33527), 229f009cdba345778e38af50ee15bb00 (127.7.67.195:46289)
I20260806 18:29:41.761324 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.757372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:42869 (local address 127.7.67.195:46289)
0806 18:29:41.757509 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:41.757513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:41.757529 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:29:41.757590 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:41.757593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:41.757640 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:41.757719 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:41.758035 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.758824 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.760291 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.760689 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.760796 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.760858 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:41.761089 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 18:29:41.761178 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:29:41.761433 9967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.756939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:33527 (local address 127.7.67.194:38003)
0806 18:29:41.757290 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:41.757304 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:41.757462 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:41.757833 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:41.757835 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:41.757851 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:41.757994 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.758000 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.758745 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.758748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.760119 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.760132 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.761027 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.761031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.761136 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.761158 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:29:41.761225 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 18:29:41.761293 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":26,"thread_start_us":117,"threads_started":1}
I20260806 18:29:41.761468 9918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.757145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:38003 (local address 127.7.67.193:33527)
0806 18:29:41.757273 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:41.757278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:41.757381 (+ 103us) server_negotiation.cc:407] Connection header received
0806 18:29:41.757611 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:41.757614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:41.757657 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:41.757695 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:41.758130 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.758628 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.760251 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.760906 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.761032 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.761085 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:41.761250 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 18:29:41.761324 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:29:41.762100 9968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.756985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:46289 (local address 127.7.67.194:42869)
0806 18:29:41.757439 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:41.757451 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:41.757518 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:41.757742 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:41.757744 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:41.757761 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:41.757926 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.757934 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.758943 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.758946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.760119 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.760132 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.760820 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.760823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.760932 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.760952 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:29:41.761925 (+ 973us) client_negotiation.cc:240] Negotiation successful
0806 18:29:41.761987 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":51,"thread_start_us":90,"threads_started":1}
I20260806 18:29:41.762539 9357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b61db7a1d4947d3aa2c92e610a2f007" candidate_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6619187e8db74ec78270cef5f3e3d223" is_pre_election: true
I20260806 18:29:41.762706 9813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b61db7a1d4947d3aa2c92e610a2f007" candidate_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229f009cdba345778e38af50ee15bb00" is_pre_election: true
I20260806 18:29:41.762817 9357 raft_consensus.cc:2454] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50cf4493c6aa44cfa0b16c5c96ead72c in term 0.
I20260806 18:29:41.762840 9813 raft_consensus.cc:2454] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50cf4493c6aa44cfa0b16c5c96ead72c in term 0.
I20260806 18:29:41.763224 9474 leader_election.cc:302] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 229f009cdba345778e38af50ee15bb00, 50cf4493c6aa44cfa0b16c5c96ead72c, 6619187e8db74ec78270cef5f3e3d223; no voters:
I20260806 18:29:41.763487 9933 raft_consensus.cc:2781] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:29:41.763548 9933 raft_consensus.cc:482] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:29:41.763592 9933 raft_consensus.cc:3037] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:41.764578 9933 raft_consensus.cc:504] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.764930 9933 leader_election.cc:288] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [CANDIDATE]: Term 1 election: Requested vote from peers 6619187e8db74ec78270cef5f3e3d223 (127.7.67.193:33527), 229f009cdba345778e38af50ee15bb00 (127.7.67.195:46289)
I20260806 18:29:41.765295 9813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b61db7a1d4947d3aa2c92e610a2f007" candidate_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229f009cdba345778e38af50ee15bb00"
I20260806 18:29:41.765326 9357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b61db7a1d4947d3aa2c92e610a2f007" candidate_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6619187e8db74ec78270cef5f3e3d223"
I20260806 18:29:41.765424 9813 raft_consensus.cc:3037] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:41.765424 9357 raft_consensus.cc:3037] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:41.766571 9357 raft_consensus.cc:2454] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50cf4493c6aa44cfa0b16c5c96ead72c in term 1.
I20260806 18:29:41.766577 9813 raft_consensus.cc:2454] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50cf4493c6aa44cfa0b16c5c96ead72c in term 1.
I20260806 18:29:41.766834 9474 leader_election.cc:302] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 229f009cdba345778e38af50ee15bb00, 50cf4493c6aa44cfa0b16c5c96ead72c; no voters:
I20260806 18:29:41.767007 9933 raft_consensus.cc:2781] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:41.767198 9933 raft_consensus.cc:686] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 1 LEADER]: Becoming Leader. State: Replica: 50cf4493c6aa44cfa0b16c5c96ead72c, State: Running, Role: LEADER
I20260806 18:29:41.767469 9933 consensus_queue.cc:237] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.769788 9107 catalog_manager.cc:5495] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c reported cstate change: term changed from 0 to 1, leader changed from <none> to 50cf4493c6aa44cfa0b16c5c96ead72c (127.7.67.194). New cstate: current_term: 1 leader_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } health_report { overall_health: UNKNOWN } } }
Aug 06 18:29:41 dist-test-slave-5bn2 krb5kdc[9035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040980, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:41.820883 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.812797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50118 (local address 127.7.67.254:42143)
0806 18:29:41.812917 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:41.812921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:41.813171 (+ 250us) server_negotiation.cc:407] Connection header received
0806 18:29:41.813298 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:41.813301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:41.813341 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:41.813405 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:41.814091 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.814609 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.815420 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.815636 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.815723 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.817931 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:41.817935 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:41.817937 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:41.817958 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:41.819272 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:41.819706 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:41.819710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:41.819711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:41.819781 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:41.820019 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:41.820023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:41.820025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:41.820203 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:41.820288 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:41.820589 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 18:29:41.820745 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":40}
I20260806 18:29:41.820883 9978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.812751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:42143 (local address 127.0.0.1:50118)
0806 18:29:41.813083 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:41.813095 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:41.813217 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:41.813443 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:41.813445 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:41.813861 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:41.813968 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.813976 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.814710 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.814713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.815306 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.815315 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.815740 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.815743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.815826 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.816386 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:41.816399 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:41.817801 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:41.819370 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:41.819380 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:41.819388 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:41.819618 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:41.819860 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:41.819863 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:41.819865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:41.819944 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:41.820304 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:41.820311 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:41.820423 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 18:29:41.820520 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 18:29:41.820686 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":143,"threads_started":1}
I20260806 18:29:41.826046 9107 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:50118:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 18:29:41.827101 9107 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:29:41.832684 9328 tablet_service.cc:1467] Processing CreateTablet for tablet 90c80babeb444bac904bca648a4b4966 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fa6db33f67aa4f1ba17b3b5db4857c31]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:29:41.832686 9778 tablet_service.cc:1467] Processing CreateTablet for tablet 90c80babeb444bac904bca648a4b4966 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fa6db33f67aa4f1ba17b3b5db4857c31]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:29:41.833087 9328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90c80babeb444bac904bca648a4b4966. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:41.833084 9778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90c80babeb444bac904bca648a4b4966. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:41.835580 9928 tablet_bootstrap.cc:492] T 90c80babeb444bac904bca648a4b4966 P 6619187e8db74ec78270cef5f3e3d223: Bootstrap starting.
I20260806 18:29:41.836090 9544 tablet_service.cc:1467] Processing CreateTablet for tablet 90c80babeb444bac904bca648a4b4966 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fa6db33f67aa4f1ba17b3b5db4857c31]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:29:41.836500 9544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90c80babeb444bac904bca648a4b4966. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:41.836630 9928 tablet_bootstrap.cc:654] T 90c80babeb444bac904bca648a4b4966 P 6619187e8db74ec78270cef5f3e3d223: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:41.836704 9923 tablet_bootstrap.cc:492] T 90c80babeb444bac904bca648a4b4966 P 229f009cdba345778e38af50ee15bb00: Bootstrap starting.
I20260806 18:29:41.837656 9923 tablet_bootstrap.cc:654] T 90c80babeb444bac904bca648a4b4966 P 229f009cdba345778e38af50ee15bb00: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:41.839087 9923 tablet_bootstrap.cc:492] T 90c80babeb444bac904bca648a4b4966 P 229f009cdba345778e38af50ee15bb00: No bootstrap required, opened a new log
I20260806 18:29:41.839175 9923 ts_tablet_manager.cc:1397] T 90c80babeb444bac904bca648a4b4966 P 229f009cdba345778e38af50ee15bb00: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:29:41.839540 9924 tablet_bootstrap.cc:492] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c: Bootstrap starting.
I20260806 18:29:41.839643 9923 raft_consensus.cc:348] T 90c80babeb444bac904bca648a4b4966 P 229f009cdba345778e38af50ee15bb00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.839762 9923 raft_consensus.cc:374] T 90c80babeb444bac904bca648a4b4966 P 229f009cdba345778e38af50ee15bb00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:41.839790 9923 raft_consensus.cc:729] T 90c80babeb444bac904bca648a4b4966 P 229f009cdba345778e38af50ee15bb00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229f009cdba345778e38af50ee15bb00, State: Initialized, Role: FOLLOWER
I20260806 18:29:41.839915 9923 consensus_queue.cc:260] T 90c80babeb444bac904bca648a4b4966 P 229f009cdba345778e38af50ee15bb00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.840134 9931 tablet_replica.cc:439] T 90c80babeb444bac904bca648a4b4966 P 229f009cdba345778e38af50ee15bb00: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } } }
I20260806 18:29:41.840253 9931 tablet_replica.cc:442] T 90c80babeb444bac904bca648a4b4966 P 229f009cdba345778e38af50ee15bb00: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:29:41.840530 9924 tablet_bootstrap.cc:654] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:41.840590 9928 tablet_bootstrap.cc:492] T 90c80babeb444bac904bca648a4b4966 P 6619187e8db74ec78270cef5f3e3d223: No bootstrap required, opened a new log
I20260806 18:29:41.840670 9928 ts_tablet_manager.cc:1397] T 90c80babeb444bac904bca648a4b4966 P 6619187e8db74ec78270cef5f3e3d223: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 18:29:41.841164 9923 ts_tablet_manager.cc:1428] T 90c80babeb444bac904bca648a4b4966 P 229f009cdba345778e38af50ee15bb00: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:29:41.841109 9928 raft_consensus.cc:348] T 90c80babeb444bac904bca648a4b4966 P 6619187e8db74ec78270cef5f3e3d223 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.841219 9928 raft_consensus.cc:374] T 90c80babeb444bac904bca648a4b4966 P 6619187e8db74ec78270cef5f3e3d223 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:41.841245 9928 raft_consensus.cc:729] T 90c80babeb444bac904bca648a4b4966 P 6619187e8db74ec78270cef5f3e3d223 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6619187e8db74ec78270cef5f3e3d223, State: Initialized, Role: FOLLOWER
I20260806 18:29:41.841347 9928 consensus_queue.cc:260] T 90c80babeb444bac904bca648a4b4966 P 6619187e8db74ec78270cef5f3e3d223 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.841638 9928 ts_tablet_manager.cc:1428] T 90c80babeb444bac904bca648a4b4966 P 6619187e8db74ec78270cef5f3e3d223: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:29:41.841594 9941 tablet_replica.cc:439] T 90c80babeb444bac904bca648a4b4966 P 6619187e8db74ec78270cef5f3e3d223: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } } }
I20260806 18:29:41.841732 9941 tablet_replica.cc:442] T 90c80babeb444bac904bca648a4b4966 P 6619187e8db74ec78270cef5f3e3d223: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:29:41.842242 9924 tablet_bootstrap.cc:492] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c: No bootstrap required, opened a new log
I20260806 18:29:41.842330 9924 ts_tablet_manager.cc:1397] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:29:41.842780 9924 raft_consensus.cc:348] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.842940 9924 raft_consensus.cc:374] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:41.842974 9924 raft_consensus.cc:729] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50cf4493c6aa44cfa0b16c5c96ead72c, State: Initialized, Role: FOLLOWER
I20260806 18:29:41.843097 9924 consensus_queue.cc:260] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.843343 9924 ts_tablet_manager.cc:1428] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:29:41.843541 9933 tablet_replica.cc:439] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } } }
I20260806 18:29:41.843668 9933 tablet_replica.cc:442] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:29:41.929906 9933 raft_consensus.cc:482] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:29:41.930063 9933 raft_consensus.cc:504] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.930548 9933 leader_election.cc:288] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6619187e8db74ec78270cef5f3e3d223 (127.7.67.193:33527), 229f009cdba345778e38af50ee15bb00 (127.7.67.195:46289)
I20260806 18:29:41.931057 9813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c80babeb444bac904bca648a4b4966" candidate_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229f009cdba345778e38af50ee15bb00" is_pre_election: true
I20260806 18:29:41.931072 9357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c80babeb444bac904bca648a4b4966" candidate_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6619187e8db74ec78270cef5f3e3d223" is_pre_election: true
I20260806 18:29:41.931228 9357 raft_consensus.cc:2454] T 90c80babeb444bac904bca648a4b4966 P 6619187e8db74ec78270cef5f3e3d223 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50cf4493c6aa44cfa0b16c5c96ead72c in term 0.
I20260806 18:29:41.931228 9813 raft_consensus.cc:2454] T 90c80babeb444bac904bca648a4b4966 P 229f009cdba345778e38af50ee15bb00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50cf4493c6aa44cfa0b16c5c96ead72c in term 0.
I20260806 18:29:41.931520 9474 leader_election.cc:302] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 229f009cdba345778e38af50ee15bb00, 50cf4493c6aa44cfa0b16c5c96ead72c; no voters:
I20260806 18:29:41.931712 9933 raft_consensus.cc:2781] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:29:41.931785 9933 raft_consensus.cc:482] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:29:41.931813 9933 raft_consensus.cc:3037] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:41.932837 9933 raft_consensus.cc:504] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.933193 9933 leader_election.cc:288] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c [CANDIDATE]: Term 1 election: Requested vote from peers 6619187e8db74ec78270cef5f3e3d223 (127.7.67.193:33527), 229f009cdba345778e38af50ee15bb00 (127.7.67.195:46289)
I20260806 18:29:41.933568 9813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c80babeb444bac904bca648a4b4966" candidate_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229f009cdba345778e38af50ee15bb00"
I20260806 18:29:41.933576 9357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c80babeb444bac904bca648a4b4966" candidate_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6619187e8db74ec78270cef5f3e3d223"
I20260806 18:29:41.933687 9357 raft_consensus.cc:3037] T 90c80babeb444bac904bca648a4b4966 P 6619187e8db74ec78270cef5f3e3d223 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:41.933692 9813 raft_consensus.cc:3037] T 90c80babeb444bac904bca648a4b4966 P 229f009cdba345778e38af50ee15bb00 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:41.934597 9813 raft_consensus.cc:2454] T 90c80babeb444bac904bca648a4b4966 P 229f009cdba345778e38af50ee15bb00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50cf4493c6aa44cfa0b16c5c96ead72c in term 1.
I20260806 18:29:41.934770 9357 raft_consensus.cc:2454] T 90c80babeb444bac904bca648a4b4966 P 6619187e8db74ec78270cef5f3e3d223 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50cf4493c6aa44cfa0b16c5c96ead72c in term 1.
I20260806 18:29:41.934926 9474 leader_election.cc:302] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 229f009cdba345778e38af50ee15bb00, 50cf4493c6aa44cfa0b16c5c96ead72c; no voters:
W20260806 18:29:41.935056 9656 tablet.cc:2369] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:29:41.935088 9933 raft_consensus.cc:2781] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:41.935178 9933 raft_consensus.cc:686] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 1 LEADER]: Becoming Leader. State: Replica: 50cf4493c6aa44cfa0b16c5c96ead72c, State: Running, Role: LEADER
I20260806 18:29:41.935333 9933 consensus_queue.cc:237] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } }
I20260806 18:29:41.936059 9970 tablet_replica.cc:439] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c: TxnStatusTablet state changed. Reason: New leader 50cf4493c6aa44cfa0b16c5c96ead72c. Latest consensus state: current_term: 1 leader_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } } }
I20260806 18:29:41.936187 9970 tablet_replica.cc:442] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:29:41.936586 9983 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 18:29:41.936800 9983 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 18:29:41.937213 9102 catalog_manager.cc:5495] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c reported cstate change: term changed from 0 to 1, leader changed from <none> to 50cf4493c6aa44cfa0b16c5c96ead72c (127.7.67.194). New cstate: current_term: 1 leader_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } health_report { overall_health: UNKNOWN } } }
I20260806 18:29:41.952530 9922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.948723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48592 (local address 127.7.67.194:42497)
0806 18:29:41.948846 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:41.948850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:41.948866 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:29:41.949078 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:41.949080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:41.949136 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:41.949217 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:29:41.949457 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.949975 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.950969 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.951215 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.951290 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.951652 (+ 362us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:29:41.952151 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:29:41.952238 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:41.952345 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 18:29:41.952401 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:29:41.952600 9978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.948613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:42497 (local address 127.0.0.1:48592)
0806 18:29:41.948802 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:41.948818 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:41.948933 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:41.949242 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:41.949246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:41.949256 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:29:41.949357 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.949363 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.950084 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.950088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.950805 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.950817 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.951328 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.951332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.951447 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.951511 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:29:41.952238 (+ 727us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:29:41.952256 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:29:41.952324 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 18:29:41.952429 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 18:29:41.955678 9813 raft_consensus.cc:1261] T 90c80babeb444bac904bca648a4b4966 P 229f009cdba345778e38af50ee15bb00 [term 1 FOLLOWER]: Refusing update from remote peer 50cf4493c6aa44cfa0b16c5c96ead72c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:29:41.955879 9357 raft_consensus.cc:1261] T 90c80babeb444bac904bca648a4b4966 P 6619187e8db74ec78270cef5f3e3d223 [term 1 FOLLOWER]: Refusing update from remote peer 50cf4493c6aa44cfa0b16c5c96ead72c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:29:41.956307 9970 consensus_queue.cc:1035] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:29:41.956496 9933 consensus_queue.cc:1035] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:29:41.958149 9931 tablet_replica.cc:439] T 90c80babeb444bac904bca648a4b4966 P 229f009cdba345778e38af50ee15bb00: TxnStatusTablet state changed. Reason: New leader 50cf4493c6aa44cfa0b16c5c96ead72c. Latest consensus state: current_term: 1 leader_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } } }
I20260806 18:29:41.958281 9931 tablet_replica.cc:442] T 90c80babeb444bac904bca648a4b4966 P 229f009cdba345778e38af50ee15bb00: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:29:41.960309 9941 tablet_replica.cc:439] T 90c80babeb444bac904bca648a4b4966 P 6619187e8db74ec78270cef5f3e3d223: TxnStatusTablet state changed. Reason: New leader 50cf4493c6aa44cfa0b16c5c96ead72c. Latest consensus state: current_term: 1 leader_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } } }
I20260806 18:29:41.960439 9941 tablet_replica.cc:442] T 90c80babeb444bac904bca648a4b4966 P 6619187e8db74ec78270cef5f3e3d223: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:29:41.961601 9970 tablet_replica.cc:439] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } } }
I20260806 18:29:41.961719 9970 tablet_replica.cc:442] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:29:41.961833 9970 tablet_replica.cc:439] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } } }
I20260806 18:29:41.961903 9970 tablet_replica.cc:442] T 90c80babeb444bac904bca648a4b4966 P 50cf4493c6aa44cfa0b16c5c96ead72c: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:29:41.962101 9949 mvcc.cc:205] Tried to move back new op lower bound from 7315623862085070848 to 7315623862008676352. Current Snapshot: MvccSnapshot[applied={T|T < 7315623862085070848 or (T in {7315623862085070848})}]
I20260806 18:29:41.962102 9931 tablet_replica.cc:439] T 90c80babeb444bac904bca648a4b4966 P 229f009cdba345778e38af50ee15bb00: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } } }
I20260806 18:29:41.962217 9931 tablet_replica.cc:442] T 90c80babeb444bac904bca648a4b4966 P 229f009cdba345778e38af50ee15bb00: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:29:41.962982 9951 mvcc.cc:205] Tried to move back new op lower bound from 7315623862085070848 to 7315623862008676352. Current Snapshot: MvccSnapshot[applied={T|T < 7315623862085070848 or (T in {7315623862085070848})}]
I20260806 18:29:41.963855 9941 tablet_replica.cc:439] T 90c80babeb444bac904bca648a4b4966 P 6619187e8db74ec78270cef5f3e3d223: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 } } peers { permanent_uuid: "50cf4493c6aa44cfa0b16c5c96ead72c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42497 } } peers { permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 } } }
I20260806 18:29:41.963987 9941 tablet_replica.cc:442] T 90c80babeb444bac904bca648a4b4966 P 6619187e8db74ec78270cef5f3e3d223: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 18:29:41.964392 9434 tablet.cc:2369] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:29:41.978713 9922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.974768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48594 (local address 127.7.67.194:42497)
0806 18:29:41.974915 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:41.974919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:41.975028 (+ 109us) server_negotiation.cc:407] Connection header received
0806 18:29:41.975177 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:41.975180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:41.975233 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:41.975309 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:29:41.975569 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.976353 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.977543 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.977894 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.977990 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.978263 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:29:41.978352 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:29:41.978410 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:41.978498 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 18:29:41.978562 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 18:29:41.990790 9922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.987616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:51713 (local address 127.7.67.194:42497)
0806 18:29:41.987742 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:41.987747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:41.987930 (+ 183us) server_negotiation.cc:407] Connection header received
0806 18:29:41.988049 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:41.988052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:41.988094 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:41.988171 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:29:41.988388 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.988924 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.989774 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.989983 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.990087 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.990355 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:29:41.990427 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:29:41.990481 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:41.990577 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 18:29:41.990619 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:29:41.990804 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:41.987558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:42497 (local address 127.7.67.194:51713)
0806 18:29:41.987843 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:41.987859 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:41.987972 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:41.988176 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:41.988179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:41.988186 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:29:41.988311 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.988317 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.989021 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.989024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.989666 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:41.989675 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:41.990107 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:41.990111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:41.990196 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:41.990239 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:29:41.990495 (+ 256us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:29:41.990501 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:29:41.990560 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 18:29:41.990635 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":108,"threads_started":1}
I20260806 18:29:41.997385 9813 raft_consensus.cc:1261] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00 [term 1 FOLLOWER]: Refusing update from remote peer 50cf4493c6aa44cfa0b16c5c96ead72c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:29:41.997821 9990 consensus_queue.cc:1035] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "229f009cdba345778e38af50ee15bb00" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:29:41.997887 9357 raft_consensus.cc:1261] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223 [term 1 FOLLOWER]: Refusing update from remote peer 50cf4493c6aa44cfa0b16c5c96ead72c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:29:41.998274 9990 consensus_queue.cc:1035] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6619187e8db74ec78270cef5f3e3d223" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:29:42.039039 9107 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:29:42.039211 9107 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:29:42.041385 9107 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45ba319e405846dea53a73b8e222f17c: Sending DeleteTablet for 3 replicas of tablet 9b61db7a1d4947d3aa2c92e610a2f007
I20260806 18:29:42.041991 9544 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b61db7a1d4947d3aa2c92e610a2f007 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:42 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:48584
I20260806 18:29:42.041999 9328 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b61db7a1d4947d3aa2c92e610a2f007 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:42 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:39256
I20260806 18:29:42.042129 9778 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b61db7a1d4947d3aa2c92e610a2f007 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:42 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:38042
I20260806 18:29:42.042212 9960 tablet_replica.cc:330] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223: stopping tablet replica
I20260806 18:29:42.042207 9959 tablet_replica.cc:330] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c: stopping tablet replica
I20260806 18:29:42.042310 9962 tablet_replica.cc:330] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00: stopping tablet replica
I20260806 18:29:42.042326 9959 raft_consensus.cc:2229] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:29:42.042328 9960 raft_consensus.cc:2229] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:29:42.042407 9962 raft_consensus.cc:2229] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:29:42.042448 9960 raft_consensus.cc:2258] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:29:42.042500 9962 raft_consensus.cc:2258] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:29:42.042662 9959 raft_consensus.cc:2258] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:29:42.043304 9962 ts_tablet_manager.cc:1905] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:29:42.043396 9959 ts_tablet_manager.cc:1905] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:29:42.044903 9960 ts_tablet_manager.cc:1905] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:29:42.046245 9959 ts_tablet_manager.cc:1918] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:29:42.046245 9962 ts_tablet_manager.cc:1918] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:29:42.046322 9962 log.cc:1196] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/9b61db7a1d4947d3aa2c92e610a2f007
I20260806 18:29:42.046322 9959 log.cc:1196] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/9b61db7a1d4947d3aa2c92e610a2f007
I20260806 18:29:42.046576 9962 ts_tablet_manager.cc:1939] T 9b61db7a1d4947d3aa2c92e610a2f007 P 229f009cdba345778e38af50ee15bb00: Deleting consensus metadata
I20260806 18:29:42.046576 9959 ts_tablet_manager.cc:1939] T 9b61db7a1d4947d3aa2c92e610a2f007 P 50cf4493c6aa44cfa0b16c5c96ead72c: Deleting consensus metadata
I20260806 18:29:42.047454 9081 catalog_manager.cc:4841] TS 50cf4493c6aa44cfa0b16c5c96ead72c (127.7.67.194:42497): tablet 9b61db7a1d4947d3aa2c92e610a2f007 (table test-table [id=98e1cca7b43e412aa493915ff8e42aff]) successfully deleted
I20260806 18:29:42.047801 9085 catalog_manager.cc:4841] TS 229f009cdba345778e38af50ee15bb00 (127.7.67.195:46289): tablet 9b61db7a1d4947d3aa2c92e610a2f007 (table test-table [id=98e1cca7b43e412aa493915ff8e42aff]) successfully deleted
I20260806 18:29:42.047964 9960 ts_tablet_manager.cc:1918] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:29:42.048050 9960 log.cc:1196] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786040973123132-7439-0/minicluster-data/ts-0/wal/wals/9b61db7a1d4947d3aa2c92e610a2f007
I20260806 18:29:42.048301 9960 ts_tablet_manager.cc:1939] T 9b61db7a1d4947d3aa2c92e610a2f007 P 6619187e8db74ec78270cef5f3e3d223: Deleting consensus metadata
I20260806 18:29:42.048998 9082 catalog_manager.cc:4841] TS 6619187e8db74ec78270cef5f3e3d223 (127.7.67.193:33527): tablet 9b61db7a1d4947d3aa2c92e610a2f007 (table test-table [id=98e1cca7b43e412aa493915ff8e42aff]) successfully deleted
Aug 06 18:29:42 dist-test-slave-5bn2 krb5kdc[9035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040981, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.67.193@KRBTEST.COM
I20260806 18:29:42.059814 9918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:42.049415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39268 (local address 127.7.67.193:33527)
0806 18:29:42.049535 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:42.049539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:42.049801 (+ 262us) server_negotiation.cc:407] Connection header received
0806 18:29:42.049938 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:42.049941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:42.049989 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:42.050059 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:42.050805 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:42.051385 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:42.052231 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:42.052458 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:42.052546 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:42.055952 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:42.055966 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:42.055982 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:42.056012 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:42.057804 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:42.058357 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:42.058381 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:42.058383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:42.058488 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:42.058756 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:42.058759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:42.058761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:42.059117 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:42.059338 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:42.059514 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 18:29:42.059646 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 18:29:42.060331 9423 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39268
I20260806 18:29:42.070135 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:42.062153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50132 (local address 127.7.67.254:42143)
0806 18:29:42.062282 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:42.062287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:42.062414 (+ 127us) server_negotiation.cc:407] Connection header received
0806 18:29:42.062551 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:42.062553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:42.062594 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:42.062658 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:42.063453 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:42.063972 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:42.064964 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:42.065169 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:42.065238 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:42.067041 (+ 1803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:42.067045 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:42.067047 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:42.067073 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:42.068423 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:42.068984 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:42.068987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:42.068988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:42.069057 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:42.069337 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:42.069339 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:42.069340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:42.069506 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:42.069607 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:42.069889 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 18:29:42.069988 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 18:29:42.070644 9107 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50132
I20260806 18:29:42.072113 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 9213
I20260806 18:29:42.091043 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 9445
I20260806 18:29:42.103852 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 9666
I20260806 18:29:42.118660 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 9059
2026-08-06T18:29:42Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4050 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:29:42 dist-test-slave-5bn2 krb5kdc[10010](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:29:42 dist-test-slave-5bn2 krb5kdc[10010](info): set up 2 sockets
Aug 06 18:29:42 dist-test-slave-5bn2 krb5kdc[10010](info): commencing operation
krb5kdc: starting...
W20260806 18:29:43.236398 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:29:43 dist-test-slave-5bn2 krb5kdc[10010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:29:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:29:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:29:43.367875 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:39361
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:37771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:39361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:43.483146 10029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:43.483460 10029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:43.483518 10029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:43.487648 10029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:29:43.487720 10029 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 18:29:43.487743 10029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:43.487767 10029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:43.487788 10029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:29:43.487814 10029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:29:43.493386 10029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:37771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:39361
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:39361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:43.494566 10029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:43.495656 10029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:43.504680 10035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:43.504680 10037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:43.504681 10034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:43.505015 10029 server_base.cc:1034] running on GCE node
I20260806 18:29:43.505618 10029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:43.506841 10029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:43.508046 10029 hybrid_clock.cc:648] HybridClock initialized: now 1786040983508022 us; error 39 us; skew 500 ppm
Aug 06 18:29:43 dist-test-slave-5bn2 krb5kdc[10010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040983, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:43.510914 10029 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:29:43.512063 10029 webserver.cc:460] Webserver started at http://127.7.67.254:34757/ using document root <none> and password file <none>
I20260806 18:29:43.512655 10029 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:43.512727 10029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:43.512935 10029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:43.515401 10029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "5c0730f3fba9473abb42ce58be6b758c"
format_stamp: "Formatted at 2026-08-06 18:29:43 on dist-test-slave-5bn2"
server_key: "0eb2fd1e02731390d9864ce36b46ebb3"
server_key_iv: "8686b9a2ebaaed623b60fa9db8a6df05"
server_key_version: "encryptionkey@0"
I20260806 18:29:43.515974 10029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "5c0730f3fba9473abb42ce58be6b758c"
format_stamp: "Formatted at 2026-08-06 18:29:43 on dist-test-slave-5bn2"
server_key: "0eb2fd1e02731390d9864ce36b46ebb3"
server_key_iv: "8686b9a2ebaaed623b60fa9db8a6df05"
server_key_version: "encryptionkey@0"
I20260806 18:29:43.519894 10029 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:29:43.522296 10049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:43.523535 10029 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:29:43.523684 10029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "5c0730f3fba9473abb42ce58be6b758c"
format_stamp: "Formatted at 2026-08-06 18:29:43 on dist-test-slave-5bn2"
server_key: "0eb2fd1e02731390d9864ce36b46ebb3"
server_key_iv: "8686b9a2ebaaed623b60fa9db8a6df05"
server_key_version: "encryptionkey@0"
I20260806 18:29:43.523789 10029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:43.538681 10029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:43.541464 10029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:43.541666 10029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:43.550299 10029 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:39361
I20260806 18:29:43.550310 10137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:39361 every 8 connection(s)
I20260806 18:29:43.551301 10029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:29:43.554105 10138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:43.555096 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 10029
I20260806 18:29:43.555209 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:29:43.555460 7439 external_mini_cluster.cc:1428] Setting key 2498d734285939baf3ac66c9416cc199
I20260806 18:29:43.560411 10138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c: Bootstrap starting.
I20260806 18:29:43.563165 10138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:43.564137 10138 log.cc:824] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:29:43 dist-test-slave-5bn2 krb5kdc[10010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:43.566349 10138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c: No bootstrap required, opened a new log
I20260806 18:29:43.569433 10138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c0730f3fba9473abb42ce58be6b758c" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39361 } }
I20260806 18:29:43.569733 10138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:43.569790 10138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0730f3fba9473abb42ce58be6b758c, State: Initialized, Role: FOLLOWER
I20260806 18:29:43.570245 10138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c0730f3fba9473abb42ce58be6b758c" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39361 } }
I20260806 18:29:43.570386 10138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:29:43.570449 10138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:29:43.570499 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:43.556849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56672 (local address 127.7.67.254:39361)
0806 18:29:43.557282 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:43.557299 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:43.557336 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:29:43.557977 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:43.557994 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:43.558334 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:43.558623 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:43.559534 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:43.560517 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:43.561685 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:43.562162 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:43.562277 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:43.565341 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:43.565347 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:43.565361 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:43.565392 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:43.567595 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:43.568032 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:43.568037 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:43.568042 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:43.568159 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:43.568428 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:43.568431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:43.568433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:43.568834 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:43.569035 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:43.569311 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 18:29:43.569606 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":135,"threads_started":1}
I20260806 18:29:43.570555 10138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:43.571565 10138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c0730f3fba9473abb42ce58be6b758c" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39361 } }
I20260806 18:29:43.571928 10138 leader_election.cc:302] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c0730f3fba9473abb42ce58be6b758c; no voters:
I20260806 18:29:43.572290 10138 leader_election.cc:288] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:29:43.572386 10144 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:43.572604 10144 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c [term 1 LEADER]: Becoming Leader. State: Replica: 5c0730f3fba9473abb42ce58be6b758c, State: Running, Role: LEADER
I20260806 18:29:43.572892 10144 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c0730f3fba9473abb42ce58be6b758c" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39361 } }
I20260806 18:29:43.573580 10138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:29:43.573993 10147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c0730f3fba9473abb42ce58be6b758c. Latest consensus state: current_term: 1 leader_uuid: "5c0730f3fba9473abb42ce58be6b758c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c0730f3fba9473abb42ce58be6b758c" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39361 } } }
I20260806 18:29:43.574129 10147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c [sys.catalog]: This master's current role is: LEADER
I20260806 18:29:43.574513 10145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c0730f3fba9473abb42ce58be6b758c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c0730f3fba9473abb42ce58be6b758c" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39361 } } }
I20260806 18:29:43.574628 10145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c [sys.catalog]: This master's current role is: LEADER
I20260806 18:29:43.575178 10149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:29:43.578573 10149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:29:43.584342 10149 catalog_manager.cc:1347] Generated new cluster ID: 967003c22c3c4e60b9f919cee675454c
I20260806 18:29:43.584434 10149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:29:43.594635 10149 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:29:43.595906 10149 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:29:43.605096 10149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c: Generated new TSK 0
I20260806 18:29:43.605942 10149 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
I20260806 18:29:43.641294 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39361
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:37771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:43.760054 10178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:43.760356 10178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:43.760434 10178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:43.764497 10178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:43.764575 10178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:43.764642 10178 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 18:29:43.764671 10178 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:29:43.764735 10178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:29:43.770121 10178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:37771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.67.254:39361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:43.771247 10178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:43.772284 10178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:43.782153 10184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:43.782153 10183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:43.782166 10186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:43.782485 10178 server_base.cc:1034] running on GCE node
I20260806 18:29:43.782979 10178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:43.783602 10178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:43.784795 10178 hybrid_clock.cc:648] HybridClock initialized: now 1786040983784774 us; error 30 us; skew 500 ppm
Aug 06 18:29:43 dist-test-slave-5bn2 krb5kdc[10010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040983, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:43.787567 10178 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:29:43.788430 10178 webserver.cc:460] Webserver started at http://127.7.67.193:36425/ using document root <none> and password file <none>
I20260806 18:29:43.788970 10178 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:43.789039 10178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:43.789247 10178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:43.791355 10178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "6e814471185d4a7285ae3f3346d41094"
format_stamp: "Formatted at 2026-08-06 18:29:43 on dist-test-slave-5bn2"
server_key: "346b0b76ca5f0c1ceddf16c25677508c"
server_key_iv: "85c7c7b0ba3d6726428786b2b3fa018b"
server_key_version: "encryptionkey@0"
I20260806 18:29:43.791903 10178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "6e814471185d4a7285ae3f3346d41094"
format_stamp: "Formatted at 2026-08-06 18:29:43 on dist-test-slave-5bn2"
server_key: "346b0b76ca5f0c1ceddf16c25677508c"
server_key_iv: "85c7c7b0ba3d6726428786b2b3fa018b"
server_key_version: "encryptionkey@0"
I20260806 18:29:43.795842 10178 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:29:43.798352 10196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:43.799608 10178 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:29:43.799723 10178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "6e814471185d4a7285ae3f3346d41094"
format_stamp: "Formatted at 2026-08-06 18:29:43 on dist-test-slave-5bn2"
server_key: "346b0b76ca5f0c1ceddf16c25677508c"
server_key_iv: "85c7c7b0ba3d6726428786b2b3fa018b"
server_key_version: "encryptionkey@0"
I20260806 18:29:43.799832 10178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:43.834383 10178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:43.837729 10178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:43.837931 10178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:43.839588 10178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:43.839643 10178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:43.839687 10178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:43.839705 10178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:29:43 dist-test-slave-5bn2 krb5kdc[10010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040983, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:43.850574 10178 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:37741
I20260806 18:29:43.850699 10375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:37741 every 8 connection(s)
I20260806 18:29:43.851768 10178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:29:43.853773 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:43.840368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:35595 (local address 127.7.67.254:39361)
0806 18:29:43.840530 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:43.840536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:43.841463 (+ 927us) server_negotiation.cc:407] Connection header received
0806 18:29:43.842690 (+ 1227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:43.842694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:43.842757 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:43.842865 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:43.844513 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:43.845131 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:43.846331 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:43.846584 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:43.846690 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:43.849971 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:43.849976 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:43.849979 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:43.850015 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:43.851647 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:43.852174 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:43.852177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:43.852179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:43.852285 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:43.852708 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:43.852710 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:43.852712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:43.852923 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:43.853006 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:43.853526 (+ 520us) server_negotiation.cc:299] Negotiation successful
0806 18:29:43.853647 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:29:43.854529 10218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:43.840743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39361 (local address 127.7.67.193:35595)
0806 18:29:43.841261 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:43.841325 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:43.842439 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:43.843087 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:43.843097 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:43.843593 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:43.844323 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:43.844347 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:43.845256 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:43.845259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:43.846201 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:43.846211 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:43.846760 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:43.846765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:43.847021 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:43.847740 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:43.847764 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:43.849777 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:43.851767 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:43.851772 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:43.851785 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:43.852068 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:43.852395 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:43.852398 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:43.852401 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:43.852587 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:43.853016 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:43.853021 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:43.853268 (+ 247us) client_negotiation.cc:769] Sending connection context
0806 18:29:43.853427 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 18:29:43.853692 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":101,"threads_started":1}
I20260806 18:29:43.855954 10376 heartbeater.cc:347] Connected to a master server at 127.7.67.254:39361
I20260806 18:29:43.856118 10376 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:43.856663 10376 heartbeater.cc:511] Master 127.7.67.254:39361 requested a full tablet report, sending...
I20260806 18:29:43.858345 10069 ts_manager.cc:194] Registered new tserver with Master: 6e814471185d4a7285ae3f3346d41094 (127.7.67.193:37741)
I20260806 18:29:43.858948 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 10178
I20260806 18:29:43.859066 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:29:43.859380 7439 external_mini_cluster.cc:1428] Setting key 1e41215ce0752636c7f53ce87c5d7aa6
I20260806 18:29:43.859778 10069 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:35595
I20260806 18:29:43.865509 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:43.858001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39361 (local address 127.7.67.193:33117)
0806 18:29:43.858368 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:43.858383 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:43.858490 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:43.858722 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:43.858725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:43.858966 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:43.859100 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:43.859108 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:43.860029 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:43.860031 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:43.860815 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:43.860825 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:43.861246 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:43.861250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:43.861389 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:43.861881 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:43.861895 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:43.862241 (+ 346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:43.864065 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:43.864068 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:43.864070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:43.864290 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:43.864564 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:43.864566 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:43.864567 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:43.864643 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:43.865041 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:43.865044 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:43.865130 (+ 86us) client_negotiation.cc:769] Sending connection context
0806 18:29:43.865213 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 18:29:43.865327 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":2,"thread_start_us":95,"threads_started":1}
I20260806 18:29:43.865550 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:43.858092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:33117 (local address 127.7.67.254:39361)
0806 18:29:43.858211 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:43.858215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:43.858460 (+ 245us) server_negotiation.cc:407] Connection header received
0806 18:29:43.858572 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:43.858574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:43.858613 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:43.858678 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:43.859212 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:43.859869 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:43.860925 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:43.861146 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:43.861234 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:43.862373 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:43.862377 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:43.862380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:43.862404 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:43.863918 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:43.864387 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:43.864391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:43.864393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:43.864483 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:43.864718 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:43.864721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:43.864723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:43.864930 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:43.865037 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:43.865285 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 18:29:43.865404 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
WARNING: no policy specified for kudu/127.7.67.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
I20260806 18:29:43.889533 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39361
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:37771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:44.004240 10388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:44.004549 10388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:44.004601 10388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:44.008673 10388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:44.008742 10388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:44.008783 10388 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 18:29:44.008810 10388 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:29:44.008859 10388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:29:44.014269 10388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:37771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.67.254:39361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:44.015527 10388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:44.016649 10388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:44.026731 10401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:44.026713 10397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:44.026772 10388 server_base.cc:1034] running on GCE node
W20260806 18:29:44.026734 10398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:44.027604 10388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:44.028307 10388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:44.029474 10388 hybrid_clock.cc:648] HybridClock initialized: now 1786040984029450 us; error 32 us; skew 500 ppm
Aug 06 18:29:44 dist-test-slave-5bn2 krb5kdc[10010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:44.032269 10388 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:29:44.033155 10388 webserver.cc:460] Webserver started at http://127.7.67.194:42257/ using document root <none> and password file <none>
I20260806 18:29:44.033735 10388 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:44.033802 10388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:44.034003 10388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:44.036135 10388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "c428d3410d224495a9c2b72f32abf12d"
format_stamp: "Formatted at 2026-08-06 18:29:44 on dist-test-slave-5bn2"
server_key: "b5050dbdf268c1bf2097f2dd53949bb3"
server_key_iv: "7160d24c4d1a87f874fd7f712555fb78"
server_key_version: "encryptionkey@0"
I20260806 18:29:44.036794 10388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "c428d3410d224495a9c2b72f32abf12d"
format_stamp: "Formatted at 2026-08-06 18:29:44 on dist-test-slave-5bn2"
server_key: "b5050dbdf268c1bf2097f2dd53949bb3"
server_key_iv: "7160d24c4d1a87f874fd7f712555fb78"
server_key_version: "encryptionkey@0"
I20260806 18:29:44.040630 10388 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:29:44.043110 10410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:44.044337 10388 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:29:44.044456 10388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "c428d3410d224495a9c2b72f32abf12d"
format_stamp: "Formatted at 2026-08-06 18:29:44 on dist-test-slave-5bn2"
server_key: "b5050dbdf268c1bf2097f2dd53949bb3"
server_key_iv: "7160d24c4d1a87f874fd7f712555fb78"
server_key_version: "encryptionkey@0"
I20260806 18:29:44.044565 10388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:44.056332 10388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:44.059623 10388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:44.059813 10388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:44.061299 10388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:44.061352 10388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:44.061406 10388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:44.061435 10388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:29:44 dist-test-slave-5bn2 krb5kdc[10010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:44.071419 10388 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:34719
I20260806 18:29:44.071506 10569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:34719 every 8 connection(s)
I20260806 18:29:44.072486 10388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:29:44.075747 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.062176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:51591 (local address 127.7.67.254:39361)
0806 18:29:44.062341 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:44.062346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:44.063229 (+ 883us) server_negotiation.cc:407] Connection header received
0806 18:29:44.064323 (+ 1094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:44.064327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:44.064393 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:44.064492 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:44.065942 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.066761 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.068439 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.068718 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.068816 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.071965 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:44.071969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:44.071971 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:44.072000 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:44.073542 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:44.074022 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:44.074025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:44.074026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:44.074116 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:44.074504 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:44.074508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:44.074510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:44.074768 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:44.074899 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:44.075403 (+ 504us) server_negotiation.cc:299] Negotiation successful
0806 18:29:44.075577 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:29:44.076537 10426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.062557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39361 (local address 127.7.67.194:51591)
0806 18:29:44.063064 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:44.063117 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:44.064089 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:44.064719 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:44.064730 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:44.065116 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:44.065787 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.065808 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.066902 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.066906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.068323 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.068335 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.068877 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.068881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.069071 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.069787 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:44.069811 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:44.071770 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:44.073665 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:44.073669 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:44.073680 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:44.073933 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:44.074198 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:44.074201 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:44.074203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:44.074378 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:44.074891 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:44.074898 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:44.075159 (+ 261us) client_negotiation.cc:769] Sending connection context
0806 18:29:44.075319 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 18:29:44.075701 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":102,"threads_started":1}
I20260806 18:29:44.076889 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 10388
I20260806 18:29:44.077004 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:29:44.077340 7439 external_mini_cluster.cc:1428] Setting key 9f2f2797d842eb950abdd8f779beb199
I20260806 18:29:44.077857 10570 heartbeater.cc:347] Connected to a master server at 127.7.67.254:39361
I20260806 18:29:44.078004 10570 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:44.078609 10570 heartbeater.cc:511] Master 127.7.67.254:39361 requested a full tablet report, sending...
I20260806 18:29:44.079702 10069 ts_manager.cc:194] Registered new tserver with Master: c428d3410d224495a9c2b72f32abf12d (127.7.67.194:34719)
I20260806 18:29:44.080559 10069 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:51591
I20260806 18:29:44.088613 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.080220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:33805 (local address 127.7.67.254:39361)
0806 18:29:44.080329 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:44.080333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:44.081666 (+ 1333us) server_negotiation.cc:407] Connection header received
0806 18:29:44.081770 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:44.081772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:44.081819 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:44.081900 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:44.082406 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.083079 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.084125 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.084324 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.084392 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.085623 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:44.085626 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:44.085628 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:44.085651 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:44.087042 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:44.087490 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:44.087492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:44.087494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:44.087575 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:44.087829 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:44.087831 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:44.087832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:44.088032 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:44.088104 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:44.088388 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 18:29:44.088480 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:29:44.088719 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.080152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39361 (local address 127.7.67.194:33805)
0806 18:29:44.081571 (+ 1419us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:44.081587 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:44.081693 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:44.081925 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:44.081928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:44.082160 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:44.082283 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.082290 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.083204 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.083207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.084017 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.084029 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.084431 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.084435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.084541 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.085002 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:44.085017 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:44.085465 (+ 448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:44.087141 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:44.087145 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:44.087147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:44.087401 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:44.087654 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:44.087657 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:44.087659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:44.087753 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:44.088125 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:44.088128 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:44.088233 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 18:29:44.088357 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 18:29:44.088521 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1327,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for kudu/127.7.67.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
I20260806 18:29:44.105381 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39361
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:37771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:44.217774 10585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:44.218052 10585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:44.218099 10585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:44.222362 10585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:44.222427 10585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:44.222466 10585 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 18:29:44.222484 10585 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:29:44.222519 10585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:29:44.227886 10585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:37771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.67.254:39361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:44.228986 10585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:44.229987 10585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:44.239643 10598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:44.239830 10585 server_base.cc:1034] running on GCE node
W20260806 18:29:44.239643 10596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:44.239653 10595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:44.240329 10585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:44.240958 10585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:44.242120 10585 hybrid_clock.cc:648] HybridClock initialized: now 1786040984242095 us; error 41 us; skew 500 ppm
Aug 06 18:29:44 dist-test-slave-5bn2 krb5kdc[10010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:44.244831 10585 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:29:44.245729 10585 webserver.cc:460] Webserver started at http://127.7.67.195:35549/ using document root <none> and password file <none>
I20260806 18:29:44.246281 10585 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:44.246344 10585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:44.246558 10585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:44.248775 10585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "75aa954eaeab4ad7957f6d1e56000cae"
format_stamp: "Formatted at 2026-08-06 18:29:44 on dist-test-slave-5bn2"
server_key: "ec3f7fb7bf31c68b59fe06a0e83aee3c"
server_key_iv: "b482fe9bf1c5e3d1bbda9cba4fde3f62"
server_key_version: "encryptionkey@0"
I20260806 18:29:44.249337 10585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "75aa954eaeab4ad7957f6d1e56000cae"
format_stamp: "Formatted at 2026-08-06 18:29:44 on dist-test-slave-5bn2"
server_key: "ec3f7fb7bf31c68b59fe06a0e83aee3c"
server_key_iv: "b482fe9bf1c5e3d1bbda9cba4fde3f62"
server_key_version: "encryptionkey@0"
I20260806 18:29:44.253291 10585 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:29:44.255833 10608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:44.257081 10585 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:29:44.257189 10585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "75aa954eaeab4ad7957f6d1e56000cae"
format_stamp: "Formatted at 2026-08-06 18:29:44 on dist-test-slave-5bn2"
server_key: "ec3f7fb7bf31c68b59fe06a0e83aee3c"
server_key_iv: "b482fe9bf1c5e3d1bbda9cba4fde3f62"
server_key_version: "encryptionkey@0"
I20260806 18:29:44.257290 10585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:44.273684 10585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:44.276597 10585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:44.276787 10585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:44.278390 10585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:44.278445 10585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:44.278510 10585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:44.278548 10585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:29:44 dist-test-slave-5bn2 krb5kdc[10010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:44.289297 10585 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:44463
I20260806 18:29:44.289347 10789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:44463 every 8 connection(s)
I20260806 18:29:44.290421 10585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:29:44.292244 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.279140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:57253 (local address 127.7.67.254:39361)
0806 18:29:44.279259 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:44.279263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:44.280373 (+ 1110us) server_negotiation.cc:407] Connection header received
0806 18:29:44.281346 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:44.281349 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:44.281416 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:44.281521 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:44.283346 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.283949 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.285208 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.285486 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.285587 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.288425 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:44.288429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:44.288431 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:44.288458 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:44.290133 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:44.290608 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:44.290612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:44.290614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:44.290712 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:44.291111 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:44.291115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:44.291116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:44.291346 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:44.291442 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:44.291941 (+ 499us) server_negotiation.cc:299] Negotiation successful
0806 18:29:44.292083 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:29:44.292243 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 10585
I20260806 18:29:44.292348 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:29:44.292636 7439 external_mini_cluster.cc:1428] Setting key c615559d951beca173d42c8ac210c416
I20260806 18:29:44.293058 10623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.279709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39361 (local address 127.7.67.195:57253)
0806 18:29:44.280229 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:44.280273 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:44.281105 (+ 832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:44.281892 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:44.281903 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:44.282411 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:44.283154 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.283179 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.284102 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.284106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.285076 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.285088 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.285667 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.285672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.285836 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.286415 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:44.286434 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:44.288245 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:44.290241 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:44.290246 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:44.290257 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:44.290504 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:44.290800 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:44.290803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:44.290805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:44.290994 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:44.291459 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:44.291466 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:44.291736 (+ 270us) client_negotiation.cc:769] Sending connection context
0806 18:29:44.291890 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 18:29:44.292148 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":156,"threads_started":1}
I20260806 18:29:44.294284 10791 heartbeater.cc:347] Connected to a master server at 127.7.67.254:39361
I20260806 18:29:44.294482 10791 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:44.295092 10791 heartbeater.cc:511] Master 127.7.67.254:39361 requested a full tablet report, sending...
I20260806 18:29:44.296123 10070 ts_manager.cc:194] Registered new tserver with Master: 75aa954eaeab4ad7957f6d1e56000cae (127.7.67.195:44463)
I20260806 18:29:44.296767 10070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:57253
I20260806 18:29:44.302629 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.296458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39361 (local address 127.7.67.195:48247)
0806 18:29:44.296763 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:44.296775 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:44.296864 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:44.297077 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:44.297079 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:44.297278 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:44.297403 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.297410 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.298135 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.298138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.298962 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.298972 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.299342 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.299344 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.299410 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.299860 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:44.299874 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:44.300201 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:44.301450 (+ 1249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:44.301452 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:44.301454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:44.301666 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:44.301899 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:44.301901 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:44.301902 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:44.301965 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:44.302246 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:44.302247 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:44.302316 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 18:29:44.302386 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 18:29:44.302486 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":78,"threads_started":1}
I20260806 18:29:44.302634 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.296531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:48247 (local address 127.7.67.254:39361)
0806 18:29:44.296651 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:44.296656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:44.296836 (+ 180us) server_negotiation.cc:407] Connection header received
0806 18:29:44.296935 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:44.296938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:44.296980 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:44.297044 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:44.297521 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.298012 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.299053 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.299258 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.299332 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.300297 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:44.300299 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:44.300301 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:44.300315 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:44.301363 (+ 1048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:44.301745 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:44.301748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:44.301750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:44.301831 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:44.302029 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:44.302031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:44.302032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:44.302183 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:44.302252 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:44.302434 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 18:29:44.302527 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:29:44.306279 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:29:44 dist-test-slave-5bn2 krb5kdc[10010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:29:44 dist-test-slave-5bn2 krb5kdc[10010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:44.331756 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.324494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56682 (local address 127.7.67.254:39361)
0806 18:29:44.324624 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:44.324629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:44.324641 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:29:44.324681 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:44.324683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:44.324723 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:44.324785 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:44.325444 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.325956 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.326709 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.326966 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.327055 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.329282 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:44.329285 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:44.329287 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:44.329311 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:44.330426 (+ 1115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:44.330793 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:44.330795 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:44.330796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:44.330906 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:44.331122 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:44.331124 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:44.331125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:44.331291 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:44.331388 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:44.331543 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 18:29:44.331633 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:29:44.334420 10070 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56682:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:29:44.337448 10070 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:29:44.351181 10818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.346226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60190 (local address 127.7.67.195:44463)
0806 18:29:44.346691 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:44.346701 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:44.346725 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:29:44.346799 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:44.346804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:44.347000 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:44.347149 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:44.347360 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.348257 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.349953 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.350499 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.350604 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.350783 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:44.350931 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 18:29:44.351022 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":152,"threads_started":1}
I20260806 18:29:44.351168 10817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.345957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:44463 (local address 127.0.0.1:60190)
0806 18:29:44.346168 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:44.346181 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:44.346669 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:44.347139 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:44.347142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:44.347161 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:44.347258 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.347265 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.348363 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.348366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.349811 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.349821 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.350616 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.350619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.350722 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.350760 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 18:29:44.350893 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 18:29:44.351021 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":50,"thread_start_us":95,"threads_started":1}
I20260806 18:29:44.351378 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.345385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:34719 (local address 127.0.0.1:39778)
0806 18:29:44.345922 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:44.345937 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:44.346024 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:44.347389 (+ 1365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:44.347392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:44.347411 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:44.347488 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.347494 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.348906 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.348910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.349838 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.349849 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.350922 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.350926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.351021 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.351044 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:29:44.351094 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 18:29:44.351161 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":115,"threads_started":1}
I20260806 18:29:44.351773 10815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.345273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:37741 (local address 127.0.0.1:56420)
0806 18:29:44.345754 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:44.345801 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:44.345925 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:44.346889 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:44.346895 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:44.346921 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:44.347075 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.347082 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.348580 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.348583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.349810 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.349821 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.350922 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.350926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.351413 (+ 487us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.351509 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 18:29:44.351579 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 18:29:44.351650 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":265,"thread_start_us":71,"threads_started":1}
I20260806 18:29:44.351866 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.345428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56420 (local address 127.7.67.193:37741)
0806 18:29:44.346359 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:44.346377 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:44.346403 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:29:44.346481 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:44.346486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:44.346712 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:44.346889 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:44.347210 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.348465 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.349935 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.350749 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.350971 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.351148 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:44.351641 (+ 493us) server_negotiation.cc:299] Negotiation successful
0806 18:29:44.351734 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":88,"threads_started":1}
I20260806 18:29:44.352558 10820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.346404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39778 (local address 127.7.67.194:34719)
0806 18:29:44.346940 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:44.346953 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:44.346973 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:29:44.347044 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:44.347049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:44.347231 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:44.347467 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:44.347619 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.348653 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.349987 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.350758 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.352118 (+ 1360us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.352287 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:44.352334 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 18:29:44.352422 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":98,"threads_started":1}
I20260806 18:29:44.354489 10683 tablet_service.cc:1467] Processing CreateTablet for tablet 65253ed402734cc0915d072cf4da3f13 (DEFAULT_TABLE table=test-table [id=5be70e0a723c479ebb68abd7dce0ba4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:44.354521 10280 tablet_service.cc:1467] Processing CreateTablet for tablet 65253ed402734cc0915d072cf4da3f13 (DEFAULT_TABLE table=test-table [id=5be70e0a723c479ebb68abd7dce0ba4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:44.354713 10480 tablet_service.cc:1467] Processing CreateTablet for tablet 65253ed402734cc0915d072cf4da3f13 (DEFAULT_TABLE table=test-table [id=5be70e0a723c479ebb68abd7dce0ba4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:44.355716 10480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65253ed402734cc0915d072cf4da3f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:44.355726 10683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65253ed402734cc0915d072cf4da3f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:44.355719 10280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65253ed402734cc0915d072cf4da3f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:44.362730 10824 tablet_bootstrap.cc:492] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d: Bootstrap starting.
I20260806 18:29:44.364066 10828 tablet_bootstrap.cc:492] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094: Bootstrap starting.
I20260806 18:29:44.364975 10824 tablet_bootstrap.cc:654] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:44.365715 10824 log.cc:824] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:29:44.366163 10828 tablet_bootstrap.cc:654] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:44.366556 10831 tablet_bootstrap.cc:492] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae: Bootstrap starting.
I20260806 18:29:44.366993 10828 log.cc:824] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094: Log is configured to *not* fsync() on all Append() calls
I20260806 18:29:44.368044 10824 tablet_bootstrap.cc:492] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d: No bootstrap required, opened a new log
I20260806 18:29:44.368269 10824 ts_tablet_manager.cc:1397] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:29:44.368670 10831 tablet_bootstrap.cc:654] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:44.368718 10828 tablet_bootstrap.cc:492] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094: No bootstrap required, opened a new log
I20260806 18:29:44.368886 10828 ts_tablet_manager.cc:1397] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:29:44.369465 10831 log.cc:824] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae: Log is configured to *not* fsync() on all Append() calls
I20260806 18:29:44.371485 10831 tablet_bootstrap.cc:492] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae: No bootstrap required, opened a new log
I20260806 18:29:44.371677 10831 ts_tablet_manager.cc:1397] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:29:44.371907 10828 raft_consensus.cc:348] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.372138 10828 raft_consensus.cc:374] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:44.372192 10828 raft_consensus.cc:729] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e814471185d4a7285ae3f3346d41094, State: Initialized, Role: FOLLOWER
I20260806 18:29:44.372071 10824 raft_consensus.cc:348] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.372334 10824 raft_consensus.cc:374] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:44.372393 10824 raft_consensus.cc:729] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c428d3410d224495a9c2b72f32abf12d, State: Initialized, Role: FOLLOWER
I20260806 18:29:44.372612 10828 consensus_queue.cc:260] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.372805 10824 consensus_queue.cc:260] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.373323 10376 heartbeater.cc:503] Master 127.7.67.254:39361 was elected leader, sending a full tablet report...
I20260806 18:29:44.373813 10828 ts_tablet_manager.cc:1428] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:29:44.374969 10824 ts_tablet_manager.cc:1428] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260806 18:29:44.375322 10570 heartbeater.cc:503] Master 127.7.67.254:39361 was elected leader, sending a full tablet report...
I20260806 18:29:44.375604 10831 raft_consensus.cc:348] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.375788 10831 raft_consensus.cc:374] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:44.375816 10831 raft_consensus.cc:729] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75aa954eaeab4ad7957f6d1e56000cae, State: Initialized, Role: FOLLOWER
I20260806 18:29:44.376137 10831 consensus_queue.cc:260] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.377076 10831 ts_tablet_manager.cc:1428] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260806 18:29:44.377372 10791 heartbeater.cc:503] Master 127.7.67.254:39361 was elected leader, sending a full tablet report...
I20260806 18:29:44.457269 10838 raft_consensus.cc:482] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:29:44.457525 10838 raft_consensus.cc:504] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.458715 10838 leader_election.cc:288] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e814471185d4a7285ae3f3346d41094 (127.7.67.193:37741), 75aa954eaeab4ad7957f6d1e56000cae (127.7.67.195:44463)
I20260806 18:29:44.463057 10818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.459128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:57001 (local address 127.7.67.195:44463)
0806 18:29:44.459283 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:44.459288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:44.459304 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:29:44.459360 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:44.459363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:44.459414 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:44.459494 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:44.459788 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.460522 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.462090 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.462501 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.462616 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.462676 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:44.462869 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 18:29:44.462943 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:29:44.463057 10426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.459046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:44463 (local address 127.7.67.194:57001)
0806 18:29:44.459163 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:44.459178 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:44.459280 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:44.459533 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:44.459535 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:44.459559 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:44.459664 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.459669 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.460652 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.460656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.461930 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.461941 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.462616 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.462620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.462732 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.462751 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:29:44.462815 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 18:29:44.462912 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 18:29:44.463733 10718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65253ed402734cc0915d072cf4da3f13" candidate_uuid: "c428d3410d224495a9c2b72f32abf12d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75aa954eaeab4ad7957f6d1e56000cae" is_pre_election: true
I20260806 18:29:44.463809 10840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.459046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:37741 (local address 127.7.67.194:33503)
0806 18:29:44.459781 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:44.459794 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:44.459899 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:44.460135 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:44.460138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:44.460155 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:44.460801 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.460810 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.461837 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.461839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.462571 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.462579 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.463389 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.463393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.463480 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.463505 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:29:44.463594 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 18:29:44.463662 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":289,"thread_start_us":166,"threads_started":1}
I20260806 18:29:44.463831 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.459128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:33503 (local address 127.7.67.193:37741)
0806 18:29:44.459253 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:44.459256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:44.459884 (+ 628us) server_negotiation.cc:407] Connection header received
0806 18:29:44.459991 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:44.459993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:44.460047 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:44.460106 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:44.460913 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.461703 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.462701 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.463157 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.463275 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.463318 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:44.463641 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 18:29:44.463717 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:29:44.464064 10718 raft_consensus.cc:2454] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c428d3410d224495a9c2b72f32abf12d in term 0.
I20260806 18:29:44.464573 10418 leader_election.cc:302] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75aa954eaeab4ad7957f6d1e56000cae, c428d3410d224495a9c2b72f32abf12d; no voters:
I20260806 18:29:44.464732 10305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65253ed402734cc0915d072cf4da3f13" candidate_uuid: "c428d3410d224495a9c2b72f32abf12d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e814471185d4a7285ae3f3346d41094" is_pre_election: true
I20260806 18:29:44.464877 10838 raft_consensus.cc:2781] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:29:44.464974 10838 raft_consensus.cc:482] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:29:44.465026 10838 raft_consensus.cc:3037] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:44.465037 10305 raft_consensus.cc:2454] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c428d3410d224495a9c2b72f32abf12d in term 0.
I20260806 18:29:44.466162 10838 raft_consensus.cc:504] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.466495 10838 leader_election.cc:288] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [CANDIDATE]: Term 1 election: Requested vote from peers 6e814471185d4a7285ae3f3346d41094 (127.7.67.193:37741), 75aa954eaeab4ad7957f6d1e56000cae (127.7.67.195:44463)
I20260806 18:29:44.466835 10718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65253ed402734cc0915d072cf4da3f13" candidate_uuid: "c428d3410d224495a9c2b72f32abf12d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75aa954eaeab4ad7957f6d1e56000cae"
I20260806 18:29:44.466840 10305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65253ed402734cc0915d072cf4da3f13" candidate_uuid: "c428d3410d224495a9c2b72f32abf12d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e814471185d4a7285ae3f3346d41094"
I20260806 18:29:44.466986 10718 raft_consensus.cc:3037] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:44.466990 10305 raft_consensus.cc:3037] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:44.468021 10305 raft_consensus.cc:2454] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c428d3410d224495a9c2b72f32abf12d in term 1.
I20260806 18:29:44.468312 10718 raft_consensus.cc:2454] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c428d3410d224495a9c2b72f32abf12d in term 1.
I20260806 18:29:44.468358 10415 leader_election.cc:302] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e814471185d4a7285ae3f3346d41094, c428d3410d224495a9c2b72f32abf12d; no voters:
I20260806 18:29:44.468608 10838 raft_consensus.cc:2781] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:44.468822 10838 raft_consensus.cc:686] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [term 1 LEADER]: Becoming Leader. State: Replica: c428d3410d224495a9c2b72f32abf12d, State: Running, Role: LEADER
I20260806 18:29:44.469132 10838 consensus_queue.cc:237] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.472306 10068 catalog_manager.cc:5495] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d reported cstate change: term changed from 0 to 1, leader changed from <none> to c428d3410d224495a9c2b72f32abf12d (127.7.67.194). New cstate: current_term: 1 leader_uuid: "c428d3410d224495a9c2b72f32abf12d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } health_report { overall_health: UNKNOWN } } }
Aug 06 18:29:44 dist-test-slave-5bn2 krb5kdc[10010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040983, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:44.520586 10851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.512703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39361 (local address 127.0.0.1:56694)
0806 18:29:44.513027 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:44.513039 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:44.513159 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:44.513380 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:44.513383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:44.513805 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:44.513901 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.513908 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.514656 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.514659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.515261 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.515270 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.515686 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.515689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.515800 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.516342 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:44.516354 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:44.517727 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:44.519083 (+ 1356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:44.519093 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:44.519100 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:44.519327 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:44.519590 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:44.519593 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:44.519595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:44.519670 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:44.520021 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:44.520028 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:44.520136 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 18:29:44.520233 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 18:29:44.520391 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":15,"thread_start_us":126,"threads_started":1}
I20260806 18:29:44.520584 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.512793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56694 (local address 127.7.67.254:39361)
0806 18:29:44.512944 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:44.512949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:44.513115 (+ 166us) server_negotiation.cc:407] Connection header received
0806 18:29:44.513243 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:44.513246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:44.513286 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:44.513350 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:44.514030 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.514556 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.515369 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.515585 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.515672 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.517862 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:44.517865 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:44.517867 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:44.517888 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:44.518994 (+ 1106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:44.519415 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:44.519419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:44.519420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:44.519510 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:44.519746 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:44.519749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:44.519751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:44.519922 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:44.520008 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:44.520293 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 18:29:44.520452 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":54}
I20260806 18:29:44.524761 10068 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:56694:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 18:29:44.525501 10068 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:29:44.530753 10480 tablet_service.cc:1467] Processing CreateTablet for tablet 9c5d54626c19410d82472a9422840573 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a65a413ee53f41a498ba0af12de264a3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:29:44.530793 10683 tablet_service.cc:1467] Processing CreateTablet for tablet 9c5d54626c19410d82472a9422840573 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a65a413ee53f41a498ba0af12de264a3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:29:44.530793 10280 tablet_service.cc:1467] Processing CreateTablet for tablet 9c5d54626c19410d82472a9422840573 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a65a413ee53f41a498ba0af12de264a3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:29:44.531529 10683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c5d54626c19410d82472a9422840573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:44.531533 10480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c5d54626c19410d82472a9422840573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:44.531529 10280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c5d54626c19410d82472a9422840573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:44.534163 10824 tablet_bootstrap.cc:492] T 9c5d54626c19410d82472a9422840573 P c428d3410d224495a9c2b72f32abf12d: Bootstrap starting.
I20260806 18:29:44.534459 10828 tablet_bootstrap.cc:492] T 9c5d54626c19410d82472a9422840573 P 6e814471185d4a7285ae3f3346d41094: Bootstrap starting.
I20260806 18:29:44.534802 10831 tablet_bootstrap.cc:492] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae: Bootstrap starting.
I20260806 18:29:44.535339 10824 tablet_bootstrap.cc:654] T 9c5d54626c19410d82472a9422840573 P c428d3410d224495a9c2b72f32abf12d: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:44.535439 10828 tablet_bootstrap.cc:654] T 9c5d54626c19410d82472a9422840573 P 6e814471185d4a7285ae3f3346d41094: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:44.535887 10831 tablet_bootstrap.cc:654] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:44.536688 10828 tablet_bootstrap.cc:492] T 9c5d54626c19410d82472a9422840573 P 6e814471185d4a7285ae3f3346d41094: No bootstrap required, opened a new log
I20260806 18:29:44.536777 10828 ts_tablet_manager.cc:1397] T 9c5d54626c19410d82472a9422840573 P 6e814471185d4a7285ae3f3346d41094: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 18:29:44.537253 10828 raft_consensus.cc:348] T 9c5d54626c19410d82472a9422840573 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.537402 10828 raft_consensus.cc:374] T 9c5d54626c19410d82472a9422840573 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:44.537426 10824 tablet_bootstrap.cc:492] T 9c5d54626c19410d82472a9422840573 P c428d3410d224495a9c2b72f32abf12d: No bootstrap required, opened a new log
I20260806 18:29:44.537438 10828 raft_consensus.cc:729] T 9c5d54626c19410d82472a9422840573 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e814471185d4a7285ae3f3346d41094, State: Initialized, Role: FOLLOWER
I20260806 18:29:44.537510 10824 ts_tablet_manager.cc:1397] T 9c5d54626c19410d82472a9422840573 P c428d3410d224495a9c2b72f32abf12d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:29:44.537557 10828 consensus_queue.cc:260] T 9c5d54626c19410d82472a9422840573 P 6e814471185d4a7285ae3f3346d41094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.537704 10831 tablet_bootstrap.cc:492] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae: No bootstrap required, opened a new log
I20260806 18:29:44.537801 10831 ts_tablet_manager.cc:1397] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:29:44.537842 10828 ts_tablet_manager.cc:1428] T 9c5d54626c19410d82472a9422840573 P 6e814471185d4a7285ae3f3346d41094: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:29:44.537946 10824 raft_consensus.cc:348] T 9c5d54626c19410d82472a9422840573 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.538056 10824 raft_consensus.cc:374] T 9c5d54626c19410d82472a9422840573 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:44.538100 10824 raft_consensus.cc:729] T 9c5d54626c19410d82472a9422840573 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c428d3410d224495a9c2b72f32abf12d, State: Initialized, Role: FOLLOWER
I20260806 18:29:44.538152 10837 tablet_replica.cc:439] T 9c5d54626c19410d82472a9422840573 P 6e814471185d4a7285ae3f3346d41094: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } } }
I20260806 18:29:44.538278 10837 tablet_replica.cc:442] T 9c5d54626c19410d82472a9422840573 P 6e814471185d4a7285ae3f3346d41094: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:29:44.538239 10824 consensus_queue.cc:260] T 9c5d54626c19410d82472a9422840573 P c428d3410d224495a9c2b72f32abf12d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.538287 10831 raft_consensus.cc:348] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.538410 10831 raft_consensus.cc:374] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:44.538451 10831 raft_consensus.cc:729] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75aa954eaeab4ad7957f6d1e56000cae, State: Initialized, Role: FOLLOWER
I20260806 18:29:44.538478 10838 tablet_replica.cc:439] T 9c5d54626c19410d82472a9422840573 P c428d3410d224495a9c2b72f32abf12d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } } }
I20260806 18:29:44.538625 10838 tablet_replica.cc:442] T 9c5d54626c19410d82472a9422840573 P c428d3410d224495a9c2b72f32abf12d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:29:44.538574 10831 consensus_queue.cc:260] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.538903 10824 ts_tablet_manager.cc:1428] T 9c5d54626c19410d82472a9422840573 P c428d3410d224495a9c2b72f32abf12d: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 18:29:44.539191 10839 tablet_replica.cc:439] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } } }
I20260806 18:29:44.539327 10839 tablet_replica.cc:442] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:29:44.539847 10831 ts_tablet_manager.cc:1428] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20260806 18:29:44.541416 10794 tablet.cc:2369] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:29:44.545471 10839 raft_consensus.cc:482] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:29:44.545576 10839 raft_consensus.cc:504] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.546573 10839 leader_election.cc:288] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e814471185d4a7285ae3f3346d41094 (127.7.67.193:37741), c428d3410d224495a9c2b72f32abf12d (127.7.67.194:34719)
I20260806 18:29:44.551283 10820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.546929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:47805 (local address 127.7.67.194:34719)
0806 18:29:44.547053 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:44.547059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:44.547361 (+ 302us) server_negotiation.cc:407] Connection header received
0806 18:29:44.547500 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:44.547504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:44.547548 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:44.547629 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:44.548020 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.548742 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.550228 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.550627 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.550744 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.550804 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:44.551085 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 18:29:44.551170 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:29:44.551301 10623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.546755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:37741 (local address 127.7.67.195:51119)
0806 18:29:44.547040 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:44.547055 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:44.547140 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:44.547469 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:44.547473 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:44.547492 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:44.547616 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.547623 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.548643 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.548646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.549930 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.549967 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.550940 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.550944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.551045 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.551070 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:29:44.551137 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 18:29:44.551198 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203}
I20260806 18:29:44.551693 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.547032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:51119 (local address 127.7.67.193:37741)
0806 18:29:44.547173 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:44.547177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:44.547190 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:29:44.547236 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:44.547239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:44.547276 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:44.547335 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:44.547736 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.548520 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.550215 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.550776 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.550958 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.551006 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:44.551477 (+ 471us) server_negotiation.cc:299] Negotiation successful
0806 18:29:44.551569 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:29:44.551308 10856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.546836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:34719 (local address 127.7.67.195:47805)
0806 18:29:44.547256 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:44.547271 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:44.547403 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:44.547649 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:44.547653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:44.547672 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:44.547834 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.547898 (+ 64us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.548853 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.548857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.550101 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.550111 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.550766 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.550771 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.550905 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.550936 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 18:29:44.551054 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 18:29:44.551137 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":104,"threads_started":1}
I20260806 18:29:44.552105 10305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c5d54626c19410d82472a9422840573" candidate_uuid: "75aa954eaeab4ad7957f6d1e56000cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e814471185d4a7285ae3f3346d41094" is_pre_election: true
I20260806 18:29:44.552256 10305 raft_consensus.cc:2454] T 9c5d54626c19410d82472a9422840573 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75aa954eaeab4ad7957f6d1e56000cae in term 0.
I20260806 18:29:44.552405 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c5d54626c19410d82472a9422840573" candidate_uuid: "75aa954eaeab4ad7957f6d1e56000cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c428d3410d224495a9c2b72f32abf12d" is_pre_election: true
I20260806 18:29:44.552664 10508 raft_consensus.cc:2454] T 9c5d54626c19410d82472a9422840573 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75aa954eaeab4ad7957f6d1e56000cae in term 0.
I20260806 18:29:44.552647 10615 leader_election.cc:302] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e814471185d4a7285ae3f3346d41094, 75aa954eaeab4ad7957f6d1e56000cae; no voters:
I20260806 18:29:44.552961 10839 raft_consensus.cc:2781] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:29:44.553020 10839 raft_consensus.cc:482] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:29:44.553052 10839 raft_consensus.cc:3037] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:44.554246 10839 raft_consensus.cc:504] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.554596 10839 leader_election.cc:288] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae [CANDIDATE]: Term 1 election: Requested vote from peers 6e814471185d4a7285ae3f3346d41094 (127.7.67.193:37741), c428d3410d224495a9c2b72f32abf12d (127.7.67.194:34719)
I20260806 18:29:44.554999 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c5d54626c19410d82472a9422840573" candidate_uuid: "75aa954eaeab4ad7957f6d1e56000cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c428d3410d224495a9c2b72f32abf12d"
I20260806 18:29:44.555004 10305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c5d54626c19410d82472a9422840573" candidate_uuid: "75aa954eaeab4ad7957f6d1e56000cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e814471185d4a7285ae3f3346d41094"
I20260806 18:29:44.555114 10305 raft_consensus.cc:3037] T 9c5d54626c19410d82472a9422840573 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:44.555119 10508 raft_consensus.cc:3037] T 9c5d54626c19410d82472a9422840573 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:44.556193 10508 raft_consensus.cc:2454] T 9c5d54626c19410d82472a9422840573 P c428d3410d224495a9c2b72f32abf12d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75aa954eaeab4ad7957f6d1e56000cae in term 1.
I20260806 18:29:44.556200 10305 raft_consensus.cc:2454] T 9c5d54626c19410d82472a9422840573 P 6e814471185d4a7285ae3f3346d41094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75aa954eaeab4ad7957f6d1e56000cae in term 1.
I20260806 18:29:44.556486 10616 leader_election.cc:302] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75aa954eaeab4ad7957f6d1e56000cae, c428d3410d224495a9c2b72f32abf12d; no voters:
I20260806 18:29:44.556701 10839 raft_consensus.cc:2781] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:44.556933 10839 raft_consensus.cc:686] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae [term 1 LEADER]: Becoming Leader. State: Replica: 75aa954eaeab4ad7957f6d1e56000cae, State: Running, Role: LEADER
I20260806 18:29:44.557267 10839 consensus_queue.cc:237] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.560099 10857 tablet_replica.cc:439] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae: TxnStatusTablet state changed. Reason: New leader 75aa954eaeab4ad7957f6d1e56000cae. Latest consensus state: current_term: 1 leader_uuid: "75aa954eaeab4ad7957f6d1e56000cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } } }
I20260806 18:29:44.560213 10857 tablet_replica.cc:442] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:29:44.560497 10860 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 18:29:44.560716 10860 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 18:29:44.560570 10070 catalog_manager.cc:5495] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae reported cstate change: term changed from 0 to 1, leader changed from <none> to 75aa954eaeab4ad7957f6d1e56000cae (127.7.67.195). New cstate: current_term: 1 leader_uuid: "75aa954eaeab4ad7957f6d1e56000cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } health_report { overall_health: HEALTHY } } }
W20260806 18:29:44.574383 10571 tablet.cc:2369] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:29:44.575532 10818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.571531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60200 (local address 127.7.67.195:44463)
0806 18:29:44.571648 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:44.571652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:44.571664 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:29:44.571859 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:44.571861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:44.571913 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:44.571992 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:29:44.572246 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.572735 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.573905 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.574126 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.574219 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.574527 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:29:44.575140 (+ 613us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:29:44.575230 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:44.575334 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 18:29:44.575385 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:29:44.575567 10851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.571410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:44463 (local address 127.0.0.1:60200)
0806 18:29:44.571599 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:44.571615 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:44.571727 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:44.572028 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:44.572031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:44.572040 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:29:44.572155 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.572162 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.572839 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.572842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.573798 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.573809 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.574229 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.574233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.574329 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.574384 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:29:44.575229 (+ 845us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:29:44.575246 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 18:29:44.575324 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 18:29:44.575414 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 18:29:44.581193 10305 raft_consensus.cc:1261] T 9c5d54626c19410d82472a9422840573 P 6e814471185d4a7285ae3f3346d41094 [term 1 FOLLOWER]: Refusing update from remote peer 75aa954eaeab4ad7957f6d1e56000cae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:29:44.581192 10508 raft_consensus.cc:1261] T 9c5d54626c19410d82472a9422840573 P c428d3410d224495a9c2b72f32abf12d [term 1 FOLLOWER]: Refusing update from remote peer 75aa954eaeab4ad7957f6d1e56000cae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:29:44.581884 10857 consensus_queue.cc:1035] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:29:44.582072 10839 consensus_queue.cc:1035] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:29:44.585199 10838 tablet_replica.cc:439] T 9c5d54626c19410d82472a9422840573 P c428d3410d224495a9c2b72f32abf12d: TxnStatusTablet state changed. Reason: New leader 75aa954eaeab4ad7957f6d1e56000cae. Latest consensus state: current_term: 1 leader_uuid: "75aa954eaeab4ad7957f6d1e56000cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } } }
I20260806 18:29:44.585330 10838 tablet_replica.cc:442] T 9c5d54626c19410d82472a9422840573 P c428d3410d224495a9c2b72f32abf12d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:29:44.586198 10837 tablet_replica.cc:439] T 9c5d54626c19410d82472a9422840573 P 6e814471185d4a7285ae3f3346d41094: TxnStatusTablet state changed. Reason: New leader 75aa954eaeab4ad7957f6d1e56000cae. Latest consensus state: current_term: 1 leader_uuid: "75aa954eaeab4ad7957f6d1e56000cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } } }
I20260806 18:29:44.586330 10837 tablet_replica.cc:442] T 9c5d54626c19410d82472a9422840573 P 6e814471185d4a7285ae3f3346d41094: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:29:44.588214 10857 tablet_replica.cc:439] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "75aa954eaeab4ad7957f6d1e56000cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } } }
I20260806 18:29:44.588342 10857 tablet_replica.cc:442] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:29:44.588459 10863 mvcc.cc:205] Tried to move back new op lower bound from 7315623872837603328 to 7315623872749039616. Current Snapshot: MvccSnapshot[applied={T|T < 7315623872837603328}]
I20260806 18:29:44.588522 10865 mvcc.cc:205] Tried to move back new op lower bound from 7315623872837603328 to 7315623872749039616. Current Snapshot: MvccSnapshot[applied={T|T < 7315623872837603328}]
I20260806 18:29:44.588819 10838 tablet_replica.cc:439] T 9c5d54626c19410d82472a9422840573 P c428d3410d224495a9c2b72f32abf12d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "75aa954eaeab4ad7957f6d1e56000cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } } }
I20260806 18:29:44.588941 10838 tablet_replica.cc:442] T 9c5d54626c19410d82472a9422840573 P c428d3410d224495a9c2b72f32abf12d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:29:44.590253 10839 tablet_replica.cc:439] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "75aa954eaeab4ad7957f6d1e56000cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } } }
I20260806 18:29:44.590358 10839 tablet_replica.cc:442] T 9c5d54626c19410d82472a9422840573 P 75aa954eaeab4ad7957f6d1e56000cae: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:29:44.591133 10837 tablet_replica.cc:439] T 9c5d54626c19410d82472a9422840573 P 6e814471185d4a7285ae3f3346d41094: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "75aa954eaeab4ad7957f6d1e56000cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } } }
I20260806 18:29:44.591236 10837 tablet_replica.cc:442] T 9c5d54626c19410d82472a9422840573 P 6e814471185d4a7285ae3f3346d41094: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:29:44.591696 10867 mvcc.cc:205] Tried to move back new op lower bound from 7315623872837603328 to 7315623872749039616. Current Snapshot: MvccSnapshot[applied={T|T < 7315623872837603328}]
I20260806 18:29:44.603379 10820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.599529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39786 (local address 127.7.67.194:34719)
0806 18:29:44.599654 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:44.599659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:44.599674 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:29:44.599808 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:44.599811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:44.599874 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:44.599976 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:29:44.600170 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.600967 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.601748 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.602094 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.602194 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.602385 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:29:44.603006 (+ 621us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:29:44.603117 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:44.603154 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 18:29:44.603215 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 18:29:44.603559 10378 tablet.cc:2369] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:29:44.615152 10818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.611731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:33299 (local address 127.7.67.195:44463)
0806 18:29:44.611862 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:44.611867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:44.611993 (+ 126us) server_negotiation.cc:407] Connection header received
0806 18:29:44.612120 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:44.612122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:44.612168 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:44.612229 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:29:44.612474 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.612962 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.614155 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.614352 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.614416 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.614751 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:29:44.614818 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:29:44.614900 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:44.614984 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 18:29:44.615036 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:29:44.615201 10882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.611592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:44463 (local address 127.7.67.194:33299)
0806 18:29:44.611914 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:44.611929 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:44.612034 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:44.612250 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:44.612253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:44.612260 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:29:44.612390 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.612396 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.613060 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.613063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.614055 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.614066 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.614453 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.614457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.614557 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.614611 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:29:44.614903 (+ 292us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:29:44.614909 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:29:44.614965 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 18:29:44.615048 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":119,"threads_started":1}
I20260806 18:29:44.622974 10305 raft_consensus.cc:1261] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094 [term 1 FOLLOWER]: Refusing update from remote peer c428d3410d224495a9c2b72f32abf12d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:29:44.623217 10718 raft_consensus.cc:1261] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae [term 1 FOLLOWER]: Refusing update from remote peer c428d3410d224495a9c2b72f32abf12d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:29:44.623754 10841 consensus_queue.cc:1035] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:29:44.623948 10838 consensus_queue.cc:1035] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [LEADER]: Connected to new peer: Peer: permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:29:44.648577 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.644706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:34719 (local address 127.7.67.195:46609)
0806 18:29:44.644873 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:44.644889 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:44.644974 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:44.645325 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:44.645327 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:44.645335 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:29:44.645453 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.645458 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.646444 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.646448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.647285 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.647296 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.647920 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.647924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.648031 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.648085 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:29:44.648351 (+ 266us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:29:44.648354 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:29:44.648394 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 18:29:44.648458 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 18:29:44.648627 10820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.644807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:46609 (local address 127.7.67.194:34719)
0806 18:29:44.645121 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:44.645125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:44.645137 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:29:44.645178 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:44.645180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:44.645228 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:44.645297 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:29:44.645555 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.646331 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.647449 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.647801 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.647904 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.648197 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:29:44.648284 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:29:44.648351 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:44.648406 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 18:29:44.648459 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238}
I20260806 18:29:44.676067 10069 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:29:44.676272 10069 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:29:44.679137 10069 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c: Sending DeleteTablet for 3 replicas of tablet 65253ed402734cc0915d072cf4da3f13
I20260806 18:29:44.679922 10280 tablet_service.cc:1514] Processing DeleteTablet for tablet 65253ed402734cc0915d072cf4da3f13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:44 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:56420
I20260806 18:29:44.680310 10480 tablet_service.cc:1514] Processing DeleteTablet for tablet 65253ed402734cc0915d072cf4da3f13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:44 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:39778
I20260806 18:29:44.680776 10683 tablet_service.cc:1514] Processing DeleteTablet for tablet 65253ed402734cc0915d072cf4da3f13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:44 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:60190
I20260806 18:29:44.680959 10894 tablet_replica.cc:330] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094: stopping tablet replica
I20260806 18:29:44.681357 10894 raft_consensus.cc:2229] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:29:44.681655 10894 raft_consensus.cc:2258] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:29:44.681856 10069 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56682:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:29:44.682294 10069 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:29:44.683867 10894 ts_tablet_manager.cc:1905] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:29:44.687242 10280 tablet_service.cc:1467] Processing CreateTablet for tablet c810580f596241838d47345c8832ba0c (DEFAULT_TABLE table=test-table [id=c4a4e53d4ee14edaab53cdcecc5b3785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:44.687511 10280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c810580f596241838d47345c8832ba0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:44.687753 10480 tablet_service.cc:1467] Processing CreateTablet for tablet c810580f596241838d47345c8832ba0c (DEFAULT_TABLE table=test-table [id=c4a4e53d4ee14edaab53cdcecc5b3785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:44.688023 10480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c810580f596241838d47345c8832ba0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:44.689822 10828 tablet_bootstrap.cc:492] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094: Bootstrap starting.
I20260806 18:29:44.690568 10683 tablet_service.cc:1467] Processing CreateTablet for tablet c810580f596241838d47345c8832ba0c (DEFAULT_TABLE table=test-table [id=c4a4e53d4ee14edaab53cdcecc5b3785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:44.690837 10683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c810580f596241838d47345c8832ba0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:44.691043 10895 tablet_replica.cc:330] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d: stopping tablet replica
I20260806 18:29:44.690838 10828 tablet_bootstrap.cc:654] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:44.691371 10895 raft_consensus.cc:2229] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:29:44.691769 10895 raft_consensus.cc:2258] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:29:44.693192 10895 ts_tablet_manager.cc:1905] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:29:44.693727 10824 tablet_bootstrap.cc:492] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d: Bootstrap starting.
I20260806 18:29:44.694720 10824 tablet_bootstrap.cc:654] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:44.696134 10895 ts_tablet_manager.cc:1918] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:29:44.696239 10895 log.cc:1196] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/65253ed402734cc0915d072cf4da3f13
I20260806 18:29:44.696372 10831 tablet_bootstrap.cc:492] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae: Bootstrap starting.
I20260806 18:29:44.696571 10895 ts_tablet_manager.cc:1939] T 65253ed402734cc0915d072cf4da3f13 P c428d3410d224495a9c2b72f32abf12d: Deleting consensus metadata
I20260806 18:29:44.697407 10831 tablet_bootstrap.cc:654] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:44.697443 10894 ts_tablet_manager.cc:1918] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:29:44.697654 10894 log.cc:1196] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/wal/wals/65253ed402734cc0915d072cf4da3f13
I20260806 18:29:44.698017 10894 ts_tablet_manager.cc:1939] T 65253ed402734cc0915d072cf4da3f13 P 6e814471185d4a7285ae3f3346d41094: Deleting consensus metadata
I20260806 18:29:44.698235 10055 catalog_manager.cc:4841] TS c428d3410d224495a9c2b72f32abf12d (127.7.67.194:34719): tablet 65253ed402734cc0915d072cf4da3f13 (table test-table [id=5be70e0a723c479ebb68abd7dce0ba4b]) successfully deleted
I20260806 18:29:44.699043 10896 tablet_replica.cc:330] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae: stopping tablet replica
I20260806 18:29:44.699399 10896 raft_consensus.cc:2229] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:29:44.699901 10054 catalog_manager.cc:4841] TS 6e814471185d4a7285ae3f3346d41094 (127.7.67.193:37741): tablet 65253ed402734cc0915d072cf4da3f13 (table test-table [id=5be70e0a723c479ebb68abd7dce0ba4b]) successfully deleted
I20260806 18:29:44.700212 10896 raft_consensus.cc:2258] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:29:44.700320 10831 tablet_bootstrap.cc:492] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae: No bootstrap required, opened a new log
I20260806 18:29:44.700440 10831 ts_tablet_manager.cc:1397] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260806 18:29:44.700891 10831 raft_consensus.cc:348] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.701038 10824 tablet_bootstrap.cc:492] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d: No bootstrap required, opened a new log
I20260806 18:29:44.701041 10831 raft_consensus.cc:374] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:44.701077 10831 raft_consensus.cc:729] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75aa954eaeab4ad7957f6d1e56000cae, State: Initialized, Role: FOLLOWER
I20260806 18:29:44.701121 10824 ts_tablet_manager.cc:1397] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260806 18:29:44.701193 10831 consensus_queue.cc:260] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.701432 10831 ts_tablet_manager.cc:1428] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:29:44.701592 10824 raft_consensus.cc:348] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.701711 10824 raft_consensus.cc:374] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:44.701745 10824 raft_consensus.cc:729] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c428d3410d224495a9c2b72f32abf12d, State: Initialized, Role: FOLLOWER
I20260806 18:29:44.701862 10824 consensus_queue.cc:260] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.702117 10824 ts_tablet_manager.cc:1428] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:29:44.705927 10896 ts_tablet_manager.cc:1905] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:29:44.706468 10828 tablet_bootstrap.cc:492] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094: No bootstrap required, opened a new log
I20260806 18:29:44.706569 10828 ts_tablet_manager.cc:1397] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.002s
I20260806 18:29:44.707094 10828 raft_consensus.cc:348] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.707219 10828 raft_consensus.cc:374] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:44.707253 10828 raft_consensus.cc:729] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e814471185d4a7285ae3f3346d41094, State: Initialized, Role: FOLLOWER
I20260806 18:29:44.707376 10828 consensus_queue.cc:260] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.707624 10828 ts_tablet_manager.cc:1428] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:29:44.710336 10896 ts_tablet_manager.cc:1918] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:29:44.710450 10896 log.cc:1196] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/65253ed402734cc0915d072cf4da3f13
I20260806 18:29:44.710826 10896 ts_tablet_manager.cc:1939] T 65253ed402734cc0915d072cf4da3f13 P 75aa954eaeab4ad7957f6d1e56000cae: Deleting consensus metadata
I20260806 18:29:44.712136 10055 catalog_manager.cc:4841] TS 75aa954eaeab4ad7957f6d1e56000cae (127.7.67.195:44463): tablet 65253ed402734cc0915d072cf4da3f13 (table test-table [id=5be70e0a723c479ebb68abd7dce0ba4b]) successfully deleted
I20260806 18:29:44.813690 10857 raft_consensus.cc:482] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:29:44.813851 10857 raft_consensus.cc:504] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.814330 10857 leader_election.cc:288] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e814471185d4a7285ae3f3346d41094 (127.7.67.193:37741), c428d3410d224495a9c2b72f32abf12d (127.7.67.194:34719)
I20260806 18:29:44.814841 10305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c810580f596241838d47345c8832ba0c" candidate_uuid: "75aa954eaeab4ad7957f6d1e56000cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e814471185d4a7285ae3f3346d41094" is_pre_election: true
I20260806 18:29:44.814894 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c810580f596241838d47345c8832ba0c" candidate_uuid: "75aa954eaeab4ad7957f6d1e56000cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c428d3410d224495a9c2b72f32abf12d" is_pre_election: true
I20260806 18:29:44.815033 10305 raft_consensus.cc:2454] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75aa954eaeab4ad7957f6d1e56000cae in term 0.
I20260806 18:29:44.815086 10508 raft_consensus.cc:2454] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75aa954eaeab4ad7957f6d1e56000cae in term 0.
I20260806 18:29:44.815363 10615 leader_election.cc:302] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e814471185d4a7285ae3f3346d41094, 75aa954eaeab4ad7957f6d1e56000cae; no voters:
I20260806 18:29:44.815572 10857 raft_consensus.cc:2781] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:29:44.815634 10857 raft_consensus.cc:482] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:29:44.815680 10857 raft_consensus.cc:3037] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:44.816735 10857 raft_consensus.cc:504] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.817077 10857 leader_election.cc:288] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [CANDIDATE]: Term 1 election: Requested vote from peers 6e814471185d4a7285ae3f3346d41094 (127.7.67.193:37741), c428d3410d224495a9c2b72f32abf12d (127.7.67.194:34719)
I20260806 18:29:44.817447 10305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c810580f596241838d47345c8832ba0c" candidate_uuid: "75aa954eaeab4ad7957f6d1e56000cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e814471185d4a7285ae3f3346d41094"
I20260806 18:29:44.817477 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c810580f596241838d47345c8832ba0c" candidate_uuid: "75aa954eaeab4ad7957f6d1e56000cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c428d3410d224495a9c2b72f32abf12d"
I20260806 18:29:44.817584 10508 raft_consensus.cc:3037] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:44.817582 10305 raft_consensus.cc:3037] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:44.818717 10508 raft_consensus.cc:2454] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75aa954eaeab4ad7957f6d1e56000cae in term 1.
I20260806 18:29:44.818717 10305 raft_consensus.cc:2454] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75aa954eaeab4ad7957f6d1e56000cae in term 1.
I20260806 18:29:44.819031 10616 leader_election.cc:302] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75aa954eaeab4ad7957f6d1e56000cae, c428d3410d224495a9c2b72f32abf12d; no voters:
I20260806 18:29:44.819212 10857 raft_consensus.cc:2781] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:44.819307 10857 raft_consensus.cc:686] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [term 1 LEADER]: Becoming Leader. State: Replica: 75aa954eaeab4ad7957f6d1e56000cae, State: Running, Role: LEADER
I20260806 18:29:44.819449 10857 consensus_queue.cc:237] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:44.821175 10070 catalog_manager.cc:5495] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae reported cstate change: term changed from 0 to 1, leader changed from <none> to 75aa954eaeab4ad7957f6d1e56000cae (127.7.67.195). New cstate: current_term: 1 leader_uuid: "75aa954eaeab4ad7957f6d1e56000cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } health_report { overall_health: HEALTHY } } }
I20260806 18:29:44.863049 10818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.859815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60210 (local address 127.7.67.195:44463)
0806 18:29:44.860135 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:44.860139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:44.860148 (+ 9us) server_negotiation.cc:407] Connection header received
0806 18:29:44.860184 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:44.860186 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:44.860225 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:44.860285 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:29:44.860532 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.861169 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.862037 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.862266 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.862352 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.862598 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:29:44.862673 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:29:44.862730 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:44.862828 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 18:29:44.862899 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251}
I20260806 18:29:44.872219 10818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.869197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:50541 (local address 127.7.67.195:44463)
0806 18:29:44.869296 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:44.869302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:44.869316 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:29:44.869469 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:44.869472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:44.869510 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:44.869578 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:29:44.869842 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.870351 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.871217 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.871427 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.871523 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.871744 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:29:44.871817 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:29:44.871891 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:44.871991 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 18:29:44.872038 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":40}
I20260806 18:29:44.872219 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:44.869109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:44463 (local address 127.7.67.195:50541)
0806 18:29:44.869248 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:44.869266 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:44.869373 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:44.869586 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:44.869589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:44.869595 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:29:44.869730 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.869738 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.870469 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.870473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.871111 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:44.871122 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:44.871530 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:44.871534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:44.871615 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:44.871640 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:29:44.871902 (+ 262us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:29:44.871909 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:29:44.871971 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 18:29:44.872057 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":22}
I20260806 18:29:44.878196 10508 raft_consensus.cc:1261] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d [term 1 FOLLOWER]: Refusing update from remote peer 75aa954eaeab4ad7957f6d1e56000cae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:29:44.878206 10305 raft_consensus.cc:1261] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094 [term 1 FOLLOWER]: Refusing update from remote peer 75aa954eaeab4ad7957f6d1e56000cae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:29:44.878607 10886 consensus_queue.cc:1035] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:29:44.878772 10857 consensus_queue.cc:1035] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:29:44.892333 10863 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 18:29:46.447191 10069 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:29:46.447352 10069 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:29:46.449994 10069 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c: Sending DeleteTablet for 3 replicas of tablet c810580f596241838d47345c8832ba0c
I20260806 18:29:46.450560 10480 tablet_service.cc:1514] Processing DeleteTablet for tablet c810580f596241838d47345c8832ba0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:46 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:39778
I20260806 18:29:46.450691 10280 tablet_service.cc:1514] Processing DeleteTablet for tablet c810580f596241838d47345c8832ba0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:46 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:56420
I20260806 18:29:46.450919 10683 tablet_service.cc:1514] Processing DeleteTablet for tablet c810580f596241838d47345c8832ba0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:46 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:60190
I20260806 18:29:46.451192 10940 tablet_replica.cc:330] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094: stopping tablet replica
I20260806 18:29:46.451372 10940 raft_consensus.cc:2229] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:29:46.451515 10940 raft_consensus.cc:2258] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:29:46.450947 10939 tablet_replica.cc:330] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d: stopping tablet replica
I20260806 18:29:46.452196 10940 ts_tablet_manager.cc:1905] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:29:46.452275 10939 raft_consensus.cc:2229] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:29:46.452421 10939 raft_consensus.cc:2258] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:29:46.453030 10939 ts_tablet_manager.cc:1905] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:29:46.454952 10940 ts_tablet_manager.cc:1918] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:29:46.455089 10940 log.cc:1196] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/wal/wals/c810580f596241838d47345c8832ba0c
I20260806 18:29:46.455078 10942 tablet_replica.cc:330] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae: stopping tablet replica
I20260806 18:29:46.455206 10942 raft_consensus.cc:2229] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:29:46.455456 10940 ts_tablet_manager.cc:1939] T c810580f596241838d47345c8832ba0c P 6e814471185d4a7285ae3f3346d41094: Deleting consensus metadata
I20260806 18:29:46.455605 10942 raft_consensus.cc:2258] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:29:46.456019 10939 ts_tablet_manager.cc:1918] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:29:46.456153 10939 log.cc:1196] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/c810580f596241838d47345c8832ba0c
I20260806 18:29:46.456172 10942 ts_tablet_manager.cc:1905] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:29:46.456231 10054 catalog_manager.cc:4841] TS 6e814471185d4a7285ae3f3346d41094 (127.7.67.193:37741): tablet c810580f596241838d47345c8832ba0c (table test-table [id=c4a4e53d4ee14edaab53cdcecc5b3785]) successfully deleted
I20260806 18:29:46.456496 10939 ts_tablet_manager.cc:1939] T c810580f596241838d47345c8832ba0c P c428d3410d224495a9c2b72f32abf12d: Deleting consensus metadata
I20260806 18:29:46.457203 10055 catalog_manager.cc:4841] TS c428d3410d224495a9c2b72f32abf12d (127.7.67.194:34719): tablet c810580f596241838d47345c8832ba0c (table test-table [id=c4a4e53d4ee14edaab53cdcecc5b3785]) successfully deleted
I20260806 18:29:46.458618 10942 ts_tablet_manager.cc:1918] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:29:46.458686 10942 log.cc:1196] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/c810580f596241838d47345c8832ba0c
I20260806 18:29:46.458976 10942 ts_tablet_manager.cc:1939] T c810580f596241838d47345c8832ba0c P 75aa954eaeab4ad7957f6d1e56000cae: Deleting consensus metadata
I20260806 18:29:46.459656 10055 catalog_manager.cc:4841] TS 75aa954eaeab4ad7957f6d1e56000cae (127.7.67.195:44463): tablet c810580f596241838d47345c8832ba0c (table test-table [id=c4a4e53d4ee14edaab53cdcecc5b3785]) successfully deleted
Aug 06 18:29:46 dist-test-slave-5bn2 krb5kdc[10010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040986, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:29:46 dist-test-slave-5bn2 krb5kdc[10010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040986, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:46.483651 10955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:46.471705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56698 (local address 127.7.67.254:39361)
0806 18:29:46.471963 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:46.471970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:46.471986 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:29:46.472099 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:46.472102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:46.472144 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:46.472227 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:46.472855 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:46.473457 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:46.474312 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:46.474558 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:46.474634 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:46.480922 (+ 6288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:46.480926 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:46.480928 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:46.480956 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:46.482227 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:46.482591 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:46.482593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:46.482595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:46.482669 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:46.482876 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:46.482879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:46.482881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:46.483101 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:46.483209 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:46.483401 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 18:29:46.483511 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":99,"threads_started":1}
I20260806 18:29:46.486232 10069 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:29:46.486685 10069 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:29:46.491221 10280 tablet_service.cc:1467] Processing CreateTablet for tablet b2012d9fda724dde8d2bff660f5f0ec9 (DEFAULT_TABLE table=test-table [id=cf16d1ca55554e87af475a0c6b1e6a5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:46.491400 10480 tablet_service.cc:1467] Processing CreateTablet for tablet b2012d9fda724dde8d2bff660f5f0ec9 (DEFAULT_TABLE table=test-table [id=cf16d1ca55554e87af475a0c6b1e6a5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:46.491570 10280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2012d9fda724dde8d2bff660f5f0ec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:46.491665 10480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2012d9fda724dde8d2bff660f5f0ec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:46.494127 10956 tablet_bootstrap.cc:492] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d: Bootstrap starting.
I20260806 18:29:46.494351 10683 tablet_service.cc:1467] Processing CreateTablet for tablet b2012d9fda724dde8d2bff660f5f0ec9 (DEFAULT_TABLE table=test-table [id=cf16d1ca55554e87af475a0c6b1e6a5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:46.494647 10683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2012d9fda724dde8d2bff660f5f0ec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:46.494679 10958 tablet_bootstrap.cc:492] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094: Bootstrap starting.
I20260806 18:29:46.495220 10956 tablet_bootstrap.cc:654] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:46.495813 10958 tablet_bootstrap.cc:654] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:46.496480 10956 tablet_bootstrap.cc:492] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d: No bootstrap required, opened a new log
I20260806 18:29:46.496588 10956 ts_tablet_manager.cc:1397] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:29:46.496974 10961 tablet_bootstrap.cc:492] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae: Bootstrap starting.
I20260806 18:29:46.497045 10956 raft_consensus.cc:348] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:46.497241 10956 raft_consensus.cc:374] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:46.497298 10956 raft_consensus.cc:729] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c428d3410d224495a9c2b72f32abf12d, State: Initialized, Role: FOLLOWER
I20260806 18:29:46.497428 10956 consensus_queue.cc:260] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:46.497823 10956 ts_tablet_manager.cc:1428] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 18:29:46.498003 10958 tablet_bootstrap.cc:492] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094: No bootstrap required, opened a new log
I20260806 18:29:46.498099 10958 ts_tablet_manager.cc:1397] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:29:46.498953 10961 tablet_bootstrap.cc:654] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:46.498764 10958 raft_consensus.cc:348] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:46.499114 10958 raft_consensus.cc:374] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:46.499161 10958 raft_consensus.cc:729] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e814471185d4a7285ae3f3346d41094, State: Initialized, Role: FOLLOWER
I20260806 18:29:46.499293 10958 consensus_queue.cc:260] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:46.499709 10958 ts_tablet_manager.cc:1428] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:29:46.500371 10961 tablet_bootstrap.cc:492] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae: No bootstrap required, opened a new log
I20260806 18:29:46.500468 10961 ts_tablet_manager.cc:1397] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260806 18:29:46.500891 10961 raft_consensus.cc:348] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:46.501016 10961 raft_consensus.cc:374] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:46.501058 10961 raft_consensus.cc:729] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75aa954eaeab4ad7957f6d1e56000cae, State: Initialized, Role: FOLLOWER
I20260806 18:29:46.501186 10961 consensus_queue.cc:260] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:46.501462 10961 ts_tablet_manager.cc:1428] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:29:46.641149 10962 raft_consensus.cc:482] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:29:46.641312 10962 raft_consensus.cc:504] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:46.641764 10962 leader_election.cc:288] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e814471185d4a7285ae3f3346d41094 (127.7.67.193:37741), 75aa954eaeab4ad7957f6d1e56000cae (127.7.67.195:44463)
I20260806 18:29:46.642299 10304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2012d9fda724dde8d2bff660f5f0ec9" candidate_uuid: "c428d3410d224495a9c2b72f32abf12d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e814471185d4a7285ae3f3346d41094" is_pre_election: true
I20260806 18:29:46.642292 10718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2012d9fda724dde8d2bff660f5f0ec9" candidate_uuid: "c428d3410d224495a9c2b72f32abf12d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75aa954eaeab4ad7957f6d1e56000cae" is_pre_election: true
I20260806 18:29:46.642442 10304 raft_consensus.cc:2454] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c428d3410d224495a9c2b72f32abf12d in term 0.
I20260806 18:29:46.642455 10718 raft_consensus.cc:2454] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c428d3410d224495a9c2b72f32abf12d in term 0.
I20260806 18:29:46.642817 10415 leader_election.cc:302] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e814471185d4a7285ae3f3346d41094, c428d3410d224495a9c2b72f32abf12d; no voters:
I20260806 18:29:46.643044 10962 raft_consensus.cc:2781] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:29:46.643112 10962 raft_consensus.cc:482] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:29:46.643138 10962 raft_consensus.cc:3037] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:46.644163 10962 raft_consensus.cc:504] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:46.644492 10962 leader_election.cc:288] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [CANDIDATE]: Term 1 election: Requested vote from peers 6e814471185d4a7285ae3f3346d41094 (127.7.67.193:37741), 75aa954eaeab4ad7957f6d1e56000cae (127.7.67.195:44463)
I20260806 18:29:46.644863 10718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2012d9fda724dde8d2bff660f5f0ec9" candidate_uuid: "c428d3410d224495a9c2b72f32abf12d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75aa954eaeab4ad7957f6d1e56000cae"
I20260806 18:29:46.644878 10304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2012d9fda724dde8d2bff660f5f0ec9" candidate_uuid: "c428d3410d224495a9c2b72f32abf12d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e814471185d4a7285ae3f3346d41094"
I20260806 18:29:46.644979 10718 raft_consensus.cc:3037] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:46.644987 10304 raft_consensus.cc:3037] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:46.646044 10718 raft_consensus.cc:2454] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c428d3410d224495a9c2b72f32abf12d in term 1.
I20260806 18:29:46.646175 10304 raft_consensus.cc:2454] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c428d3410d224495a9c2b72f32abf12d in term 1.
I20260806 18:29:46.646391 10418 leader_election.cc:302] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75aa954eaeab4ad7957f6d1e56000cae, c428d3410d224495a9c2b72f32abf12d; no voters:
I20260806 18:29:46.646638 10962 raft_consensus.cc:2781] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:46.646896 10962 raft_consensus.cc:686] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [term 1 LEADER]: Becoming Leader. State: Replica: c428d3410d224495a9c2b72f32abf12d, State: Running, Role: LEADER
I20260806 18:29:46.647066 10962 consensus_queue.cc:237] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:46.648743 10070 catalog_manager.cc:5495] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d reported cstate change: term changed from 0 to 1, leader changed from <none> to c428d3410d224495a9c2b72f32abf12d (127.7.67.194). New cstate: current_term: 1 leader_uuid: "c428d3410d224495a9c2b72f32abf12d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } health_report { overall_health: UNKNOWN } } }
I20260806 18:29:46.695760 10972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:46.692558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60236 (local address 127.7.67.194:34719)
0806 18:29:46.692850 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:46.692857 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:46.692868 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:29:46.692905 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:46.692907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:46.692949 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:46.693020 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:29:46.693290 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:46.693819 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:46.694680 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:46.694945 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:46.695036 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:46.695311 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:29:46.695389 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:29:46.695450 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:46.695543 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 18:29:46.695605 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":123,"threads_started":1}
I20260806 18:29:46.702764 10304 raft_consensus.cc:1261] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094 [term 1 FOLLOWER]: Refusing update from remote peer c428d3410d224495a9c2b72f32abf12d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:29:46.702764 10718 raft_consensus.cc:1261] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae [term 1 FOLLOWER]: Refusing update from remote peer c428d3410d224495a9c2b72f32abf12d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:29:46.703197 10962 consensus_queue.cc:1035] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:29:46.703405 10965 consensus_queue.cc:1035] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [LEADER]: Connected to new peer: Peer: permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:29:46.735033 10070 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:29:46.735196 10070 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:29:46.737190 10070 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c: Sending DeleteTablet for 3 replicas of tablet b2012d9fda724dde8d2bff660f5f0ec9
I20260806 18:29:46.737766 10280 tablet_service.cc:1514] Processing DeleteTablet for tablet b2012d9fda724dde8d2bff660f5f0ec9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:46 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:56420
I20260806 18:29:46.737866 10480 tablet_service.cc:1514] Processing DeleteTablet for tablet b2012d9fda724dde8d2bff660f5f0ec9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:46 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:39778
I20260806 18:29:46.737991 10940 tablet_replica.cc:330] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094: stopping tablet replica
I20260806 18:29:46.738057 10683 tablet_service.cc:1514] Processing DeleteTablet for tablet b2012d9fda724dde8d2bff660f5f0ec9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:46 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:60190
I20260806 18:29:46.738122 10939 tablet_replica.cc:330] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d: stopping tablet replica
I20260806 18:29:46.738127 10940 raft_consensus.cc:2229] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:29:46.738242 10942 tablet_replica.cc:330] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae: stopping tablet replica
I20260806 18:29:46.738252 10940 raft_consensus.cc:2258] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:29:46.738265 10939 raft_consensus.cc:2229] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:29:46.738359 10942 raft_consensus.cc:2229] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:29:46.738471 10942 raft_consensus.cc:2258] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:29:46.738461 10939 raft_consensus.cc:2258] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:29:46.738960 10070 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 18:29:46.739120 10940 ts_tablet_manager.cc:1905] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:29:46.739290 10939 ts_tablet_manager.cc:1905] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 18:29:46.739399 10070 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:29:46.740483 10942 ts_tablet_manager.cc:1905] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:29:46.741573 10939 ts_tablet_manager.cc:1918] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:29:46.741644 10939 log.cc:1196] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/b2012d9fda724dde8d2bff660f5f0ec9
I20260806 18:29:46.741680 10940 ts_tablet_manager.cc:1918] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:29:46.741767 10940 log.cc:1196] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-0/wal/wals/b2012d9fda724dde8d2bff660f5f0ec9
I20260806 18:29:46.741901 10939 ts_tablet_manager.cc:1939] T b2012d9fda724dde8d2bff660f5f0ec9 P c428d3410d224495a9c2b72f32abf12d: Deleting consensus metadata
I20260806 18:29:46.742007 10940 ts_tablet_manager.cc:1939] T b2012d9fda724dde8d2bff660f5f0ec9 P 6e814471185d4a7285ae3f3346d41094: Deleting consensus metadata
I20260806 18:29:46.742615 10055 catalog_manager.cc:4841] TS c428d3410d224495a9c2b72f32abf12d (127.7.67.194:34719): tablet b2012d9fda724dde8d2bff660f5f0ec9 (table test-table [id=cf16d1ca55554e87af475a0c6b1e6a5c]) successfully deleted
I20260806 18:29:46.742682 10054 catalog_manager.cc:4841] TS 6e814471185d4a7285ae3f3346d41094 (127.7.67.193:37741): tablet b2012d9fda724dde8d2bff660f5f0ec9 (table test-table [id=cf16d1ca55554e87af475a0c6b1e6a5c]) successfully deleted
I20260806 18:29:46.743096 10942 ts_tablet_manager.cc:1918] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:29:46.743181 10942 log.cc:1196] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/b2012d9fda724dde8d2bff660f5f0ec9
I20260806 18:29:46.743419 10942 ts_tablet_manager.cc:1939] T b2012d9fda724dde8d2bff660f5f0ec9 P 75aa954eaeab4ad7957f6d1e56000cae: Deleting consensus metadata
I20260806 18:29:46.744017 10055 catalog_manager.cc:4841] TS 75aa954eaeab4ad7957f6d1e56000cae (127.7.67.195:44463): tablet b2012d9fda724dde8d2bff660f5f0ec9 (table test-table [id=cf16d1ca55554e87af475a0c6b1e6a5c]) successfully deleted
I20260806 18:29:46.745635 10280 tablet_service.cc:1467] Processing CreateTablet for tablet 0805db3c68d8488f84c6d9e938de7403 (DEFAULT_TABLE table=test-table [id=562fbc4c83fe4f3586d1f2c1410f7b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:46.745635 10480 tablet_service.cc:1467] Processing CreateTablet for tablet 0805db3c68d8488f84c6d9e938de7403 (DEFAULT_TABLE table=test-table [id=562fbc4c83fe4f3586d1f2c1410f7b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:46.745718 10683 tablet_service.cc:1467] Processing CreateTablet for tablet 0805db3c68d8488f84c6d9e938de7403 (DEFAULT_TABLE table=test-table [id=562fbc4c83fe4f3586d1f2c1410f7b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:46.745898 10480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0805db3c68d8488f84c6d9e938de7403. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:46.745914 10280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0805db3c68d8488f84c6d9e938de7403. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:46.745980 10683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0805db3c68d8488f84c6d9e938de7403. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:46.748258 10956 tablet_bootstrap.cc:492] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d: Bootstrap starting.
I20260806 18:29:46.748324 10961 tablet_bootstrap.cc:492] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae: Bootstrap starting.
I20260806 18:29:46.749150 10956 tablet_bootstrap.cc:654] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:46.749166 10961 tablet_bootstrap.cc:654] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:46.749619 10958 tablet_bootstrap.cc:492] T 0805db3c68d8488f84c6d9e938de7403 P 6e814471185d4a7285ae3f3346d41094: Bootstrap starting.
I20260806 18:29:46.750125 10961 tablet_bootstrap.cc:492] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae: No bootstrap required, opened a new log
I20260806 18:29:46.750188 10961 ts_tablet_manager.cc:1397] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 18:29:46.750386 10958 tablet_bootstrap.cc:654] T 0805db3c68d8488f84c6d9e938de7403 P 6e814471185d4a7285ae3f3346d41094: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:46.750571 10961 raft_consensus.cc:348] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:46.750658 10961 raft_consensus.cc:374] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:46.750679 10961 raft_consensus.cc:729] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75aa954eaeab4ad7957f6d1e56000cae, State: Initialized, Role: FOLLOWER
I20260806 18:29:46.750895 10961 consensus_queue.cc:260] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:46.751256 10961 ts_tablet_manager.cc:1428] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:29:46.751704 10958 tablet_bootstrap.cc:492] T 0805db3c68d8488f84c6d9e938de7403 P 6e814471185d4a7285ae3f3346d41094: No bootstrap required, opened a new log
I20260806 18:29:46.751791 10958 ts_tablet_manager.cc:1397] T 0805db3c68d8488f84c6d9e938de7403 P 6e814471185d4a7285ae3f3346d41094: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:29:46.752151 10956 tablet_bootstrap.cc:492] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d: No bootstrap required, opened a new log
I20260806 18:29:46.752247 10956 ts_tablet_manager.cc:1397] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 18:29:46.752228 10958 raft_consensus.cc:348] T 0805db3c68d8488f84c6d9e938de7403 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:46.752342 10958 raft_consensus.cc:374] T 0805db3c68d8488f84c6d9e938de7403 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:46.752377 10958 raft_consensus.cc:729] T 0805db3c68d8488f84c6d9e938de7403 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e814471185d4a7285ae3f3346d41094, State: Initialized, Role: FOLLOWER
I20260806 18:29:46.752503 10958 consensus_queue.cc:260] T 0805db3c68d8488f84c6d9e938de7403 P 6e814471185d4a7285ae3f3346d41094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:46.752650 10956 raft_consensus.cc:348] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:46.752751 10958 ts_tablet_manager.cc:1428] T 0805db3c68d8488f84c6d9e938de7403 P 6e814471185d4a7285ae3f3346d41094: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:29:46.752771 10956 raft_consensus.cc:374] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:46.752811 10956 raft_consensus.cc:729] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c428d3410d224495a9c2b72f32abf12d, State: Initialized, Role: FOLLOWER
I20260806 18:29:46.752936 10956 consensus_queue.cc:260] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:46.753189 10956 ts_tablet_manager.cc:1428] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:29:46.883584 10911 raft_consensus.cc:482] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:29:46.883750 10911 raft_consensus.cc:504] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:46.884191 10911 leader_election.cc:288] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e814471185d4a7285ae3f3346d41094 (127.7.67.193:37741), c428d3410d224495a9c2b72f32abf12d (127.7.67.194:34719)
I20260806 18:29:46.884720 10304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0805db3c68d8488f84c6d9e938de7403" candidate_uuid: "75aa954eaeab4ad7957f6d1e56000cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e814471185d4a7285ae3f3346d41094" is_pre_election: true
I20260806 18:29:46.884727 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0805db3c68d8488f84c6d9e938de7403" candidate_uuid: "75aa954eaeab4ad7957f6d1e56000cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c428d3410d224495a9c2b72f32abf12d" is_pre_election: true
I20260806 18:29:46.884872 10507 raft_consensus.cc:2454] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75aa954eaeab4ad7957f6d1e56000cae in term 0.
I20260806 18:29:46.884882 10304 raft_consensus.cc:2454] T 0805db3c68d8488f84c6d9e938de7403 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75aa954eaeab4ad7957f6d1e56000cae in term 0.
I20260806 18:29:46.885196 10615 leader_election.cc:302] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e814471185d4a7285ae3f3346d41094, 75aa954eaeab4ad7957f6d1e56000cae; no voters:
I20260806 18:29:46.885377 10911 raft_consensus.cc:2781] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:29:46.885442 10911 raft_consensus.cc:482] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:29:46.885473 10911 raft_consensus.cc:3037] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:46.886476 10911 raft_consensus.cc:504] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:46.886798 10911 leader_election.cc:288] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [CANDIDATE]: Term 1 election: Requested vote from peers 6e814471185d4a7285ae3f3346d41094 (127.7.67.193:37741), c428d3410d224495a9c2b72f32abf12d (127.7.67.194:34719)
I20260806 18:29:46.887173 10304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0805db3c68d8488f84c6d9e938de7403" candidate_uuid: "75aa954eaeab4ad7957f6d1e56000cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e814471185d4a7285ae3f3346d41094"
I20260806 18:29:46.887187 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0805db3c68d8488f84c6d9e938de7403" candidate_uuid: "75aa954eaeab4ad7957f6d1e56000cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c428d3410d224495a9c2b72f32abf12d"
I20260806 18:29:46.887283 10304 raft_consensus.cc:3037] T 0805db3c68d8488f84c6d9e938de7403 P 6e814471185d4a7285ae3f3346d41094 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:46.887300 10507 raft_consensus.cc:3037] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:46.888216 10304 raft_consensus.cc:2454] T 0805db3c68d8488f84c6d9e938de7403 P 6e814471185d4a7285ae3f3346d41094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75aa954eaeab4ad7957f6d1e56000cae in term 1.
I20260806 18:29:46.888331 10507 raft_consensus.cc:2454] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75aa954eaeab4ad7957f6d1e56000cae in term 1.
I20260806 18:29:46.888485 10615 leader_election.cc:302] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e814471185d4a7285ae3f3346d41094, 75aa954eaeab4ad7957f6d1e56000cae; no voters:
I20260806 18:29:46.888708 10911 raft_consensus.cc:2781] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:46.888798 10911 raft_consensus.cc:686] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [term 1 LEADER]: Becoming Leader. State: Replica: 75aa954eaeab4ad7957f6d1e56000cae, State: Running, Role: LEADER
I20260806 18:29:46.888934 10911 consensus_queue.cc:237] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } }
I20260806 18:29:46.890667 10069 catalog_manager.cc:5495] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae reported cstate change: term changed from 0 to 1, leader changed from <none> to 75aa954eaeab4ad7957f6d1e56000cae (127.7.67.195). New cstate: current_term: 1 leader_uuid: "75aa954eaeab4ad7957f6d1e56000cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75aa954eaeab4ad7957f6d1e56000cae" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44463 } health_report { overall_health: HEALTHY } } }
I20260806 18:29:46.908557 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:46.905373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42222 (local address 127.7.67.195:44463)
0806 18:29:46.905688 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:46.905695 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:46.905705 (+ 10us) server_negotiation.cc:407] Connection header received
0806 18:29:46.905740 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:46.905742 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:46.905781 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:46.905854 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:29:46.906114 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:46.906649 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:46.907525 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:46.907752 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:46.907843 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:46.908096 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:29:46.908171 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:29:46.908227 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:46.908336 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 18:29:46.908392 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":136,"threads_started":1}
I20260806 18:29:46.914505 10304 raft_consensus.cc:1261] T 0805db3c68d8488f84c6d9e938de7403 P 6e814471185d4a7285ae3f3346d41094 [term 1 FOLLOWER]: Refusing update from remote peer 75aa954eaeab4ad7957f6d1e56000cae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:29:46.914830 10507 raft_consensus.cc:1261] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d [term 1 FOLLOWER]: Refusing update from remote peer 75aa954eaeab4ad7957f6d1e56000cae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:29:46.914942 10923 consensus_queue.cc:1035] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e814471185d4a7285ae3f3346d41094" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 37741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:29:46.915241 10911 consensus_queue.cc:1035] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "c428d3410d224495a9c2b72f32abf12d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 34719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:29:46.928646 10908 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 18:29:48.476209 10070 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:29:48.476373 10070 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:29:48.478722 10070 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c0730f3fba9473abb42ce58be6b758c: Sending DeleteTablet for 3 replicas of tablet 0805db3c68d8488f84c6d9e938de7403
I20260806 18:29:48.479353 10682 tablet_service.cc:1514] Processing DeleteTablet for tablet 0805db3c68d8488f84c6d9e938de7403 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:48 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:60190
I20260806 18:29:48.479347 10280 tablet_service.cc:1514] Processing DeleteTablet for tablet 0805db3c68d8488f84c6d9e938de7403 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:48 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:56420
I20260806 18:29:48.479707 10480 tablet_service.cc:1514] Processing DeleteTablet for tablet 0805db3c68d8488f84c6d9e938de7403 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:48 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:39778
I20260806 18:29:48.479789 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 10178
I20260806 18:29:48.488144 11026 tablet_replica.cc:330] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d: stopping tablet replica
I20260806 18:29:48.488299 11026 raft_consensus.cc:2229] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:29:48.488420 11026 raft_consensus.cc:2258] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:29:48.489117 11026 ts_tablet_manager.cc:1905] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:29:48.491060 11024 tablet_replica.cc:330] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae: stopping tablet replica
I20260806 18:29:48.491218 11024 raft_consensus.cc:2229] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:29:48.491482 11024 raft_consensus.cc:2258] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:29:48.491609 11026 ts_tablet_manager.cc:1918] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:29:48.491700 11026 log.cc:1196] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/0805db3c68d8488f84c6d9e938de7403
I20260806 18:29:48.491987 11026 ts_tablet_manager.cc:1939] T 0805db3c68d8488f84c6d9e938de7403 P c428d3410d224495a9c2b72f32abf12d: Deleting consensus metadata
I20260806 18:29:48.492246 11024 ts_tablet_manager.cc:1905] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:29:48.492787 10055 catalog_manager.cc:4841] TS c428d3410d224495a9c2b72f32abf12d (127.7.67.194:34719): tablet 0805db3c68d8488f84c6d9e938de7403 (table test-table [id=562fbc4c83fe4f3586d1f2c1410f7b12]) successfully deleted
I20260806 18:29:48.494530 11024 ts_tablet_manager.cc:1918] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:29:48.494603 11024 log.cc:1196] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/0805db3c68d8488f84c6d9e938de7403
I20260806 18:29:48.494833 11024 ts_tablet_manager.cc:1939] T 0805db3c68d8488f84c6d9e938de7403 P 75aa954eaeab4ad7957f6d1e56000cae: Deleting consensus metadata
I20260806 18:29:48.495591 10055 catalog_manager.cc:4841] TS 75aa954eaeab4ad7957f6d1e56000cae (127.7.67.195:44463): tablet 0805db3c68d8488f84c6d9e938de7403 (table test-table [id=562fbc4c83fe4f3586d1f2c1410f7b12]) successfully deleted
W20260806 18:29:48.495786 10054 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.193:37741 (error 108)
I20260806 18:29:48.496630 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 10388
I20260806 18:29:48.512779 11028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:48.496465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:37741 (local address 127.0.0.1:35976)
0806 18:29:48.496712 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:48.512654 (+ 15942us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:37741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":81,"threads_started":1}
W20260806 18:29:48.512979 10054 catalog_manager.cc:4584] TS 6e814471185d4a7285ae3f3346d41094 (127.7.67.193:37741): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0805db3c68d8488f84c6d9e938de7403: Network error: Client connection negotiation failed: client connection to 127.7.67.193:37741: connect: Connection refused (error 111)
I20260806 18:29:48.519868 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 10585
I20260806 18:29:48.535066 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 10029
2026-08-06T18:29:48Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6416 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:29:48 dist-test-slave-5bn2 krb5kdc[11032](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:29:48 dist-test-slave-5bn2 krb5kdc[11032](info): set up 2 sockets
Aug 06 18:29:48 dist-test-slave-5bn2 krb5kdc[11032](info): commencing operation
krb5kdc: starting...
W20260806 18:29:49.663203 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:29:49 dist-test-slave-5bn2 krb5kdc[11032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:29:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:29:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:29:49.787164 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:35591
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:40309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:35591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:49.901320 11049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:49.901602 11049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:49.901643 11049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:49.905623 11049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:29:49.905684 11049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:49.905702 11049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:49.905717 11049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:29:49.905736 11049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:29:49.910950 11049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:40309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:35591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:35591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:49.911945 11049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:49.912915 11049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:49.921403 11060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:49.921473 11049 server_base.cc:1034] running on GCE node
W20260806 18:29:49.921571 11058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:49.921502 11063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:49.922293 11049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:49.923450 11049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:49.924636 11049 hybrid_clock.cc:648] HybridClock initialized: now 1786040989924619 us; error 30 us; skew 500 ppm
Aug 06 18:29:49 dist-test-slave-5bn2 krb5kdc[11032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:49.927405 11049 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:29:49.928339 11049 webserver.cc:460] Webserver started at http://127.7.67.254:46759/ using document root <none> and password file <none>
I20260806 18:29:49.928897 11049 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:49.928965 11049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:49.929159 11049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:49.931300 11049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "d37311ce94674cefaf992a8a889adb46"
format_stamp: "Formatted at 2026-08-06 18:29:49 on dist-test-slave-5bn2"
server_key: "9d5b20f218b3d76c8bbb1eec073573a0"
server_key_iv: "077ed6115a4c8ac0ffac28a5a6a6acd2"
server_key_version: "encryptionkey@0"
I20260806 18:29:49.931852 11049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "d37311ce94674cefaf992a8a889adb46"
format_stamp: "Formatted at 2026-08-06 18:29:49 on dist-test-slave-5bn2"
server_key: "9d5b20f218b3d76c8bbb1eec073573a0"
server_key_iv: "077ed6115a4c8ac0ffac28a5a6a6acd2"
server_key_version: "encryptionkey@0"
I20260806 18:29:49.935482 11049 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:29:49.937805 11074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:49.939002 11049 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:29:49.939116 11049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "d37311ce94674cefaf992a8a889adb46"
format_stamp: "Formatted at 2026-08-06 18:29:49 on dist-test-slave-5bn2"
server_key: "9d5b20f218b3d76c8bbb1eec073573a0"
server_key_iv: "077ed6115a4c8ac0ffac28a5a6a6acd2"
server_key_version: "encryptionkey@0"
I20260806 18:29:49.939209 11049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:49.977699 11049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:49.980551 11049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:49.980741 11049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:49.987782 11049 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:35591
I20260806 18:29:49.987804 11155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:35591 every 8 connection(s)
I20260806 18:29:49.988744 11049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:29:49.991602 11156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:49.993511 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 11049
I20260806 18:29:49.993629 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:29:49.993916 7439 external_mini_cluster.cc:1428] Setting key b7710ad83299fd46a19134c62d1f598a
I20260806 18:29:49.998304 11156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46: Bootstrap starting.
I20260806 18:29:50.000908 11156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:50.001823 11156 log.cc:824] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:29:50 dist-test-slave-5bn2 krb5kdc[11032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:50.003932 11156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46: No bootstrap required, opened a new log
I20260806 18:29:50.007589 11156 raft_consensus.cc:348] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37311ce94674cefaf992a8a889adb46" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 35591 } }
I20260806 18:29:50.007815 11156 raft_consensus.cc:374] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:50.007848 11156 raft_consensus.cc:729] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d37311ce94674cefaf992a8a889adb46, State: Initialized, Role: FOLLOWER
I20260806 18:29:50.008070 11161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:49.995343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56996 (local address 127.7.67.254:35591)
0806 18:29:49.995743 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:49.995758 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:49.995794 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:29:49.996509 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:49.996528 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:49.996857 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:49.997135 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:49.998025 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:49.999068 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:50.000229 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:50.000703 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:50.000818 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:50.003504 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:50.003510 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:50.003524 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:50.003551 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:50.005289 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:50.005701 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:50.005706 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:50.005711 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:50.005823 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:50.006086 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:50.006089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:50.006091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:50.006482 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:50.006699 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:50.006993 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 18:29:50.007273 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":107,"threads_started":1}
I20260806 18:29:50.008306 11156 consensus_queue.cc:260] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37311ce94674cefaf992a8a889adb46" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 35591 } }
I20260806 18:29:50.008442 11156 raft_consensus.cc:388] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:29:50.008507 11156 raft_consensus.cc:482] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:29:50.008605 11156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:50.009492 11156 raft_consensus.cc:504] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37311ce94674cefaf992a8a889adb46" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 35591 } }
I20260806 18:29:50.009815 11156 leader_election.cc:302] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d37311ce94674cefaf992a8a889adb46; no voters:
I20260806 18:29:50.010177 11156 leader_election.cc:288] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:29:50.010268 11163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:50.010496 11163 raft_consensus.cc:686] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46 [term 1 LEADER]: Becoming Leader. State: Replica: d37311ce94674cefaf992a8a889adb46, State: Running, Role: LEADER
I20260806 18:29:50.010834 11163 consensus_queue.cc:237] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37311ce94674cefaf992a8a889adb46" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 35591 } }
I20260806 18:29:50.011427 11156 sys_catalog.cc:564] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:29:50.012598 11166 sys_catalog.cc:455] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d37311ce94674cefaf992a8a889adb46. Latest consensus state: current_term: 1 leader_uuid: "d37311ce94674cefaf992a8a889adb46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37311ce94674cefaf992a8a889adb46" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 35591 } } }
I20260806 18:29:50.012619 11165 sys_catalog.cc:455] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d37311ce94674cefaf992a8a889adb46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37311ce94674cefaf992a8a889adb46" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 35591 } } }
I20260806 18:29:50.012737 11166 sys_catalog.cc:458] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46 [sys.catalog]: This master's current role is: LEADER
I20260806 18:29:50.012738 11165 sys_catalog.cc:458] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46 [sys.catalog]: This master's current role is: LEADER
I20260806 18:29:50.013113 11177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:29:50.015900 11177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:29:50.021485 11177 catalog_manager.cc:1347] Generated new cluster ID: cf27f3285dd146efa8aa3446f075705e
I20260806 18:29:50.021562 11177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:29:50.039743 11177 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:29:50.040719 11177 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:29:50.047082 11177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d37311ce94674cefaf992a8a889adb46: Generated new TSK 0
I20260806 18:29:50.047776 11177 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
I20260806 18:29:50.082206 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:35591
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:40309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:50.196228 11194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:50.196508 11194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:50.196554 11194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:50.200616 11194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:50.200695 11194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:50.200753 11194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:29:50.206212 11194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:40309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:35591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:50.207373 11194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:50.208454 11194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:50.218230 11200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:50.218389 11194 server_base.cc:1034] running on GCE node
W20260806 18:29:50.218336 11203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:50.218247 11201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:50.218976 11194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:50.219627 11194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:50.220781 11194 hybrid_clock.cc:648] HybridClock initialized: now 1786040990220759 us; error 37 us; skew 500 ppm
Aug 06 18:29:50 dist-test-slave-5bn2 krb5kdc[11032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040990, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:50.223536 11194 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:29:50.224406 11194 webserver.cc:460] Webserver started at http://127.7.67.193:33239/ using document root <none> and password file <none>
I20260806 18:29:50.224944 11194 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:50.225003 11194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:50.225178 11194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:50.227252 11194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "da88975baf16453c81996dd595f3b924"
format_stamp: "Formatted at 2026-08-06 18:29:50 on dist-test-slave-5bn2"
server_key: "66d405be45e1b9cfb04a5f9a14751a76"
server_key_iv: "6c621d21cece83fd4d0ddbe16811695e"
server_key_version: "encryptionkey@0"
I20260806 18:29:50.227771 11194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "da88975baf16453c81996dd595f3b924"
format_stamp: "Formatted at 2026-08-06 18:29:50 on dist-test-slave-5bn2"
server_key: "66d405be45e1b9cfb04a5f9a14751a76"
server_key_iv: "6c621d21cece83fd4d0ddbe16811695e"
server_key_version: "encryptionkey@0"
I20260806 18:29:50.231525 11194 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:29:50.234102 11217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:50.235296 11194 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:29:50.235395 11194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "da88975baf16453c81996dd595f3b924"
format_stamp: "Formatted at 2026-08-06 18:29:50 on dist-test-slave-5bn2"
server_key: "66d405be45e1b9cfb04a5f9a14751a76"
server_key_iv: "6c621d21cece83fd4d0ddbe16811695e"
server_key_version: "encryptionkey@0"
I20260806 18:29:50.235481 11194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:50.250406 11194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:50.253270 11194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:50.253437 11194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:50.254104 11194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:50.255074 11194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:50.255123 11194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:50.255164 11194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:50.255185 11194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:50.263688 11194 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:45139
I20260806 18:29:50.263731 11397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:45139 every 8 connection(s)
I20260806 18:29:50.264587 11194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:29:50.269258 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 11194
I20260806 18:29:50.269357 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:29:50.269606 7439 external_mini_cluster.cc:1428] Setting key 4cfe2f946fcb93e59a6075b03e5f305c
Aug 06 18:29:50 dist-test-slave-5bn2 krb5kdc[11032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040990, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:50.278023 11161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:50.266371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:38269 (local address 127.7.67.254:35591)
0806 18:29:50.266558 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:50.266563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:50.267674 (+ 1111us) server_negotiation.cc:407] Connection header received
0806 18:29:50.268521 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:50.268524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:50.268582 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:50.268654 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:50.269887 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:50.270536 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:50.271738 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:50.271963 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:50.272051 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:50.274670 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:50.274674 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:50.274676 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:50.274700 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:50.276050 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:50.276538 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:50.276541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:50.276543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:50.276630 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:50.276946 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:50.276949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:50.276951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:50.277134 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:50.277220 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:50.277724 (+ 504us) server_negotiation.cc:299] Negotiation successful
0806 18:29:50.277852 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 18:29:50.278836 11402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:50.266774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:35591 (local address 127.7.67.193:38269)
0806 18:29:50.267524 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:50.267569 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:50.268348 (+ 779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:50.268809 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:50.268816 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:50.269186 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:50.269736 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:50.269751 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:50.270672 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:50.270675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:50.271618 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:50.271628 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:50.272114 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:50.272119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:50.272275 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:50.272857 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:50.272880 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:50.274507 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:50.276164 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:50.276170 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:50.276185 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:50.276442 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:50.276714 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:50.276718 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:50.276720 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:50.276859 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:50.277236 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:50.277243 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:50.277491 (+ 248us) client_negotiation.cc:769] Sending connection context
0806 18:29:50.277664 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 18:29:50.277992 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":129,"threads_started":1}
WARNING: no policy specified for kudu/127.7.67.194@KRBTEST.COM; defaulting to no policy
I20260806 18:29:50.280046 11398 heartbeater.cc:347] Connected to a master server at 127.7.67.254:35591
I20260806 18:29:50.280295 11398 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:50.280809 11398 heartbeater.cc:511] Master 127.7.67.254:35591 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.194@KRBTEST.COM" created.
I20260806 18:29:50.282470 11099 ts_manager.cc:194] Registered new tserver with Master: da88975baf16453c81996dd595f3b924 (127.7.67.193:45139)
I20260806 18:29:50.283926 11099 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:38269
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
I20260806 18:29:50.297322 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:35591
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:40309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:50.411625 11406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:50.411947 11406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:50.412021 11406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:50.416450 11406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:50.416517 11406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:50.416599 11406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:29:50.421991 11406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:40309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:35591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:50.423185 11406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:50.424263 11406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:50.434157 11413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:50.434170 11416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:50.434173 11424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:50.434424 11406 server_base.cc:1034] running on GCE node
I20260806 18:29:50.434831 11406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:50.435497 11406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:50.436681 11406 hybrid_clock.cc:648] HybridClock initialized: now 1786040990436659 us; error 32 us; skew 500 ppm
Aug 06 18:29:50 dist-test-slave-5bn2 krb5kdc[11032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040990, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:50.439599 11406 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:29:50.440470 11406 webserver.cc:460] Webserver started at http://127.7.67.194:45711/ using document root <none> and password file <none>
I20260806 18:29:50.441023 11406 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:50.441092 11406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:50.441298 11406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:50.443521 11406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "6314d5aaaf6b4f0f898b0f7d73dde566"
format_stamp: "Formatted at 2026-08-06 18:29:50 on dist-test-slave-5bn2"
server_key: "f83a158a6c945309c28a82e8d75ca4f6"
server_key_iv: "04fa31468af03464f646943fa1855b8b"
server_key_version: "encryptionkey@0"
I20260806 18:29:50.444075 11406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "6314d5aaaf6b4f0f898b0f7d73dde566"
format_stamp: "Formatted at 2026-08-06 18:29:50 on dist-test-slave-5bn2"
server_key: "f83a158a6c945309c28a82e8d75ca4f6"
server_key_iv: "04fa31468af03464f646943fa1855b8b"
server_key_version: "encryptionkey@0"
I20260806 18:29:50.447733 11406 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:29:50.450121 11432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:50.451324 11406 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:29:50.451448 11406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "6314d5aaaf6b4f0f898b0f7d73dde566"
format_stamp: "Formatted at 2026-08-06 18:29:50 on dist-test-slave-5bn2"
server_key: "f83a158a6c945309c28a82e8d75ca4f6"
server_key_iv: "04fa31468af03464f646943fa1855b8b"
server_key_version: "encryptionkey@0"
I20260806 18:29:50.451540 11406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:50.472692 11406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:50.476042 11406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:50.476234 11406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:50.476858 11406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:50.477713 11406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:50.477762 11406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:50.477818 11406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:50.477846 11406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:50.486294 11406 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:43601
I20260806 18:29:50.486340 11593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:43601 every 8 connection(s)
I20260806 18:29:50.487236 11406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:29:50.494709 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 11406
I20260806 18:29:50.494833 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:29:50.495127 7439 external_mini_cluster.cc:1428] Setting key d2103fa046be7923e8a0a8c2fd768edc
Aug 06 18:29:50 dist-test-slave-5bn2 krb5kdc[11032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040990, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:50.500825 11161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:50.489075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:33491 (local address 127.7.67.254:35591)
0806 18:29:50.489206 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:50.489210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:50.490295 (+ 1085us) server_negotiation.cc:407] Connection header received
0806 18:29:50.491255 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:50.491257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:50.491306 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:50.491379 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:50.492551 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:50.493091 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:50.494178 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:50.494436 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:50.494541 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:50.497244 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:50.497248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:50.497250 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:50.497275 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:50.498662 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:50.499263 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:50.499267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:50.499269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:50.499362 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:50.499689 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:50.499692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:50.499694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:50.499926 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:50.500025 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:50.500536 (+ 511us) server_negotiation.cc:299] Negotiation successful
0806 18:29:50.500663 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:29:50.501633 11598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:50.489436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:35591 (local address 127.7.67.194:33491)
0806 18:29:50.490137 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:50.490182 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:50.491060 (+ 878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:50.491536 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:50.491543 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:50.491907 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:50.492402 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:50.492415 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:50.493182 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:50.493184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:50.494065 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:50.494074 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:50.494594 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:50.494597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:50.494749 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:50.495387 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:50.495403 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:50.497073 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:50.498809 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:50.498814 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:50.498827 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:50.499157 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:50.499443 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:50.499446 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:50.499448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:50.499592 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:50.500015 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:50.500020 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:50.500269 (+ 249us) client_negotiation.cc:769] Sending connection context
0806 18:29:50.500429 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 18:29:50.500706 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":116,"threads_started":1}
I20260806 18:29:50.502811 11595 heartbeater.cc:347] Connected to a master server at 127.7.67.254:35591
I20260806 18:29:50.503116 11595 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:50.503754 11595 heartbeater.cc:511] Master 127.7.67.254:35591 requested a full tablet report, sending...
I20260806 18:29:50.504927 11099 ts_manager.cc:194] Registered new tserver with Master: 6314d5aaaf6b4f0f898b0f7d73dde566 (127.7.67.194:43601)
I20260806 18:29:50.505704 11099 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:33491
WARNING: no policy specified for kudu/127.7.67.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
I20260806 18:29:50.525456 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:35591
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:40309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:50.639858 11601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:50.640151 11601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:50.640202 11601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:50.644389 11601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:50.644459 11601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:50.644531 11601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:29:50.649973 11601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:40309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:35591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:50.651177 11601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:50.652235 11601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:29:50.662151 11601 server_base.cc:1034] running on GCE node
W20260806 18:29:50.662113 11616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:50.662137 11613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:50.662113 11614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:50.662787 11601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:50.663473 11601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:50.664630 11601 hybrid_clock.cc:648] HybridClock initialized: now 1786040990664605 us; error 39 us; skew 500 ppm
Aug 06 18:29:50 dist-test-slave-5bn2 krb5kdc[11032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040990, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:50.667375 11601 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:29:50.668277 11601 webserver.cc:460] Webserver started at http://127.7.67.195:40315/ using document root <none> and password file <none>
I20260806 18:29:50.668810 11601 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:50.668866 11601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:50.669054 11601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:50.671180 11601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "910b4225d5f14de8ac4a0a3e14c366c6"
format_stamp: "Formatted at 2026-08-06 18:29:50 on dist-test-slave-5bn2"
server_key: "448940ca50295f8308b6f87b938c3789"
server_key_iv: "78cea8323b2edfd6dadb50f1f81aaf83"
server_key_version: "encryptionkey@0"
I20260806 18:29:50.671697 11601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "910b4225d5f14de8ac4a0a3e14c366c6"
format_stamp: "Formatted at 2026-08-06 18:29:50 on dist-test-slave-5bn2"
server_key: "448940ca50295f8308b6f87b938c3789"
server_key_iv: "78cea8323b2edfd6dadb50f1f81aaf83"
server_key_version: "encryptionkey@0"
I20260806 18:29:50.675473 11601 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260806 18:29:50.677928 11627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:50.679272 11601 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:29:50.679390 11601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "910b4225d5f14de8ac4a0a3e14c366c6"
format_stamp: "Formatted at 2026-08-06 18:29:50 on dist-test-slave-5bn2"
server_key: "448940ca50295f8308b6f87b938c3789"
server_key_iv: "78cea8323b2edfd6dadb50f1f81aaf83"
server_key_version: "encryptionkey@0"
I20260806 18:29:50.679499 11601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:50.691558 11601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:50.694658 11601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:50.694842 11601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:50.695482 11601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:50.696305 11601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:50.696355 11601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:50.696398 11601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:50.696427 11601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:50.705089 11601 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:35701
I20260806 18:29:50.705114 11810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:35701 every 8 connection(s)
I20260806 18:29:50.706017 11601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:29:50.712646 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 11601
I20260806 18:29:50.712776 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:29:50.713078 7439 external_mini_cluster.cc:1428] Setting key 6ea36ae07a0375a9229cd251b9a61da3
Aug 06 18:29:50 dist-test-slave-5bn2 krb5kdc[11032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040990, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:50.719758 11161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:50.707973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:49959 (local address 127.7.67.254:35591)
0806 18:29:50.708153 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:50.708156 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:50.709152 (+ 996us) server_negotiation.cc:407] Connection header received
0806 18:29:50.710025 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:50.710028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:50.710087 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:50.710150 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:50.711376 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:50.711995 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:50.713179 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:50.713527 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:50.713630 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:50.716739 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:50.716742 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:50.716745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:50.716771 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:50.717911 (+ 1140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:50.718345 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:50.718347 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:50.718349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:50.718460 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:50.718792 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:50.718794 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:50.718795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:50.719002 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:50.719079 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:50.719525 (+ 446us) server_negotiation.cc:299] Negotiation successful
0806 18:29:50.719631 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 18:29:50.720582 11815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:50.708313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:35591 (local address 127.7.67.195:49959)
0806 18:29:50.709005 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:50.709047 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:50.709831 (+ 784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:50.710298 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:50.710305 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:50.710673 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:50.711238 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:50.711253 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:50.712104 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:50.712107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:50.713064 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:50.713075 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:50.713818 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:50.713821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:50.714035 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:50.714722 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:50.714773 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:50.716571 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:50.718007 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:50.718012 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:50.718024 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:50.718272 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:50.718540 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:50.718543 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:50.718545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:50.718721 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:50.719081 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:50.719086 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:50.719331 (+ 245us) client_negotiation.cc:769] Sending connection context
0806 18:29:50.719491 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 18:29:50.719747 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":121,"threads_started":1}
I20260806 18:29:50.721716 11813 heartbeater.cc:347] Connected to a master server at 127.7.67.254:35591
I20260806 18:29:50.721946 11813 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:50.722415 11813 heartbeater.cc:511] Master 127.7.67.254:35591 requested a full tablet report, sending...
I20260806 18:29:50.723438 11099 ts_manager.cc:194] Registered new tserver with Master: 910b4225d5f14de8ac4a0a3e14c366c6 (127.7.67.195:35701)
I20260806 18:29:50.724001 11099 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:49959
I20260806 18:29:50.727636 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 18:29:50.742223 11161 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:29:50.740840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57002 (local address 127.7.67.254:35591)
0806 18:29:50.740966 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:50.740970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:50.741115 (+ 145us) server_negotiation.cc:407] Connection header received
0806 18:29:50.741228 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:50.741231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:50.741274 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:50.741352 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:50.742065 (+ 713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57002: BlockingRecv error: recv got EOF from 127.0.0.1:57002 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:29:50.743383 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 11194
I20260806 18:29:50.755039 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 11406
I20260806 18:29:50.767441 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 11601
I20260806 18:29:50.778064 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 11049
2026-08-06T18:29:50Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2243 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:29:50 dist-test-slave-5bn2 krb5kdc[11825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:29:50 dist-test-slave-5bn2 krb5kdc[11825](info): set up 2 sockets
Aug 06 18:29:50 dist-test-slave-5bn2 krb5kdc[11825](info): commencing operation
krb5kdc: starting...
W20260806 18:29:51.903728 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:29:51 dist-test-slave-5bn2 krb5kdc[11825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:29:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:29:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:29:52.033144 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:39057
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:43789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:39057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:52.146802 11846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:52.147125 11846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:52.147166 11846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:52.151194 11846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:29:52.151257 11846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:52.151275 11846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:52.151292 11846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:29:52.151306 11846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:29:52.156761 11846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:43789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:39057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:39057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:52.157873 11846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:52.158946 11846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:52.167161 11852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:52.167181 11853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:52.167176 11855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:52.167615 11846 server_base.cc:1034] running on GCE node
I20260806 18:29:52.168220 11846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:52.169361 11846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:52.170521 11846 hybrid_clock.cc:648] HybridClock initialized: now 1786040992170504 us; error 28 us; skew 500 ppm
Aug 06 18:29:52 dist-test-slave-5bn2 krb5kdc[11825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040992, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:52.173249 11846 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:29:52.174108 11846 webserver.cc:460] Webserver started at http://127.7.67.254:41227/ using document root <none> and password file <none>
I20260806 18:29:52.174628 11846 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:52.174685 11846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:52.174906 11846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:52.176924 11846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "6f39835e5a4b478bb1adad66d72e10d6"
format_stamp: "Formatted at 2026-08-06 18:29:52 on dist-test-slave-5bn2"
server_key: "67ceefb49371b7b2376674cf04a0797a"
server_key_iv: "858892ffbe87dc0a4b745ea263b441a8"
server_key_version: "encryptionkey@0"
I20260806 18:29:52.177456 11846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "6f39835e5a4b478bb1adad66d72e10d6"
format_stamp: "Formatted at 2026-08-06 18:29:52 on dist-test-slave-5bn2"
server_key: "67ceefb49371b7b2376674cf04a0797a"
server_key_iv: "858892ffbe87dc0a4b745ea263b441a8"
server_key_version: "encryptionkey@0"
I20260806 18:29:52.181140 11846 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:29:52.183487 11865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:52.184671 11846 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:29:52.184792 11846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "6f39835e5a4b478bb1adad66d72e10d6"
format_stamp: "Formatted at 2026-08-06 18:29:52 on dist-test-slave-5bn2"
server_key: "67ceefb49371b7b2376674cf04a0797a"
server_key_iv: "858892ffbe87dc0a4b745ea263b441a8"
server_key_version: "encryptionkey@0"
I20260806 18:29:52.184890 11846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:52.205806 11846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:52.208706 11846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:52.208890 11846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:52.216039 11846 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:39057
I20260806 18:29:52.216329 11954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:39057 every 8 connection(s)
I20260806 18:29:52.217029 11846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:29:52.219905 11955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:52.219951 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 11846
I20260806 18:29:52.220054 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:29:52.220326 7439 external_mini_cluster.cc:1428] Setting key 4de4c59eb95b9d981d4c5ee52e8a5350
I20260806 18:29:52.226403 11955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6: Bootstrap starting.
I20260806 18:29:52.228736 11955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6: Neither blocks nor log segments found. Creating new log.
Aug 06 18:29:52 dist-test-slave-5bn2 krb5kdc[11825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:52.229502 11955 log.cc:824] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:29:52.231521 11955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6: No bootstrap required, opened a new log
I20260806 18:29:52.234364 11960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:52.221752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60986 (local address 127.7.67.254:39057)
0806 18:29:52.222196 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:52.222212 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:52.222248 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:29:52.223180 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:52.223202 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:52.223558 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:52.223878 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:52.224706 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:52.225698 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:52.226559 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:52.227084 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:52.227233 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:52.229620 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:52.229627 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:52.229645 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:52.229678 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:52.231278 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:52.231697 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:52.231703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:52.231709 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:52.231864 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:52.232125 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:52.232128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:52.232130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:52.232647 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:52.232823 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:52.233039 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 18:29:52.233356 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":126,"threads_started":1}
I20260806 18:29:52.236008 11955 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f39835e5a4b478bb1adad66d72e10d6" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39057 } }
I20260806 18:29:52.236266 11955 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:52.236370 11955 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f39835e5a4b478bb1adad66d72e10d6, State: Initialized, Role: FOLLOWER
I20260806 18:29:52.236840 11955 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f39835e5a4b478bb1adad66d72e10d6" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39057 } }
I20260806 18:29:52.236976 11955 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:29:52.237036 11955 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:29:52.237133 11955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:52.237283 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:29:52.238049 11955 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f39835e5a4b478bb1adad66d72e10d6" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39057 } }
I20260806 18:29:52.238374 11955 leader_election.cc:302] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f39835e5a4b478bb1adad66d72e10d6; no voters:
I20260806 18:29:52.238670 11955 leader_election.cc:288] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:29:52.238772 11964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:52.238978 11964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6f39835e5a4b478bb1adad66d72e10d6, State: Running, Role: LEADER
I20260806 18:29:52.239356 11964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f39835e5a4b478bb1adad66d72e10d6" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39057 } }
I20260806 18:29:52.240020 11955 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:29:52.241088 11965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f39835e5a4b478bb1adad66d72e10d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f39835e5a4b478bb1adad66d72e10d6" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39057 } } }
I20260806 18:29:52.241223 11965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:29:52.241456 11965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f39835e5a4b478bb1adad66d72e10d6. Latest consensus state: current_term: 1 leader_uuid: "6f39835e5a4b478bb1adad66d72e10d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f39835e5a4b478bb1adad66d72e10d6" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39057 } } }
I20260806 18:29:52.241545 11965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6 [sys.catalog]: This master's current role is: LEADER
W20260806 18:29:52.245328 11987 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:29:52.245422 11987 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:29:52.245493 11977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:29:52.246218 11977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:29:52.252542 11977 catalog_manager.cc:1347] Generated new cluster ID: ee2f37389d5e4908a90ae9e7dc8c4be0
I20260806 18:29:52.252617 11977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:29:52.257752 11960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:52.251831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60988 (local address 127.7.67.254:39057)
0806 18:29:52.251953 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:52.251958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:52.252286 (+ 328us) server_negotiation.cc:407] Connection header received
0806 18:29:52.252421 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:52.252424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:52.252477 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:52.252581 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:52.253338 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:52.254144 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:52.255397 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:52.255738 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:52.255822 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:52.256773 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:52.256777 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:52.256779 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:29:52.257300 (+ 521us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:52.257334 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:29:52.257353 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:52.257433 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:52.257547 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 18:29:52.257608 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:29:52.263274 11977 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:29:52.264361 11977 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:29:52.275074 11977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f39835e5a4b478bb1adad66d72e10d6: Generated new TSK 0
I20260806 18:29:52.275929 11977 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:29:52.283916 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 11846
2026-08-06T18:29:52Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1506 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:29:52 dist-test-slave-5bn2 krb5kdc[12001](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:29:52 dist-test-slave-5bn2 krb5kdc[12001](info): set up 2 sockets
Aug 06 18:29:52 dist-test-slave-5bn2 krb5kdc[12001](info): commencing operation
krb5kdc: starting...
W20260806 18:29:53.402213 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:29:53 dist-test-slave-5bn2 krb5kdc[12001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:29:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:29:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:29:53.526810 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:41705
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:41705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:53.639943 12019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:53.640237 12019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:53.640293 12019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:53.644327 12019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:29:53.644395 12019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:53.644423 12019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:53.644443 12019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:29:53.644469 12019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:29:53.649863 12019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:41705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:41705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:53.651103 12019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:53.652164 12019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:53.660863 12024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:53.660840 12030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:53.660897 12019 server_base.cc:1034] running on GCE node
W20260806 18:29:53.660835 12025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:53.661612 12019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:53.662801 12019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:53.663983 12019 hybrid_clock.cc:648] HybridClock initialized: now 1786040993663959 us; error 41 us; skew 500 ppm
Aug 06 18:29:53 dist-test-slave-5bn2 krb5kdc[12001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040993, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:53.666746 12019 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:29:53.667708 12019 webserver.cc:460] Webserver started at http://127.7.67.254:39041/ using document root <none> and password file <none>
I20260806 18:29:53.668272 12019 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:53.668340 12019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:53.668546 12019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:53.670632 12019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "5e2eaa2d8f764513a524d5747d159c9e"
format_stamp: "Formatted at 2026-08-06 18:29:53 on dist-test-slave-5bn2"
server_key: "d6eda010fa0432d92d0c8b28fed50c78"
server_key_iv: "a1c2775fe4af7c940d8e092c2c8bbaa6"
server_key_version: "encryptionkey@0"
I20260806 18:29:53.671247 12019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "5e2eaa2d8f764513a524d5747d159c9e"
format_stamp: "Formatted at 2026-08-06 18:29:53 on dist-test-slave-5bn2"
server_key: "d6eda010fa0432d92d0c8b28fed50c78"
server_key_iv: "a1c2775fe4af7c940d8e092c2c8bbaa6"
server_key_version: "encryptionkey@0"
I20260806 18:29:53.675026 12019 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:29:53.677412 12042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:53.678633 12019 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:29:53.678751 12019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "5e2eaa2d8f764513a524d5747d159c9e"
format_stamp: "Formatted at 2026-08-06 18:29:53 on dist-test-slave-5bn2"
server_key: "d6eda010fa0432d92d0c8b28fed50c78"
server_key_iv: "a1c2775fe4af7c940d8e092c2c8bbaa6"
server_key_version: "encryptionkey@0"
I20260806 18:29:53.678875 12019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:53.693750 12019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:53.696677 12019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:53.696870 12019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:53.703831 12019 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:41705
I20260806 18:29:53.704113 12121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:41705 every 8 connection(s)
I20260806 18:29:53.704823 12019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:29:53.707669 12122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:53.713310 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 12019
I20260806 18:29:53.713439 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:29:53.713474 12122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e: Bootstrap starting.
I20260806 18:29:53.713708 7439 external_mini_cluster.cc:1428] Setting key fcc78a3ad02e18f30726a102d4ff2652
I20260806 18:29:53.716156 12122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:53.717041 12122 log.cc:824] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e: Log is configured to *not* fsync() on all Append() calls
I20260806 18:29:53.719352 12122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e: No bootstrap required, opened a new log
Aug 06 18:29:53 dist-test-slave-5bn2 krb5kdc[12001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:53.723554 12122 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2eaa2d8f764513a524d5747d159c9e" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 41705 } }
I20260806 18:29:53.723804 12122 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:53.723898 12122 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2eaa2d8f764513a524d5747d159c9e, State: Initialized, Role: FOLLOWER
I20260806 18:29:53.724416 12122 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2eaa2d8f764513a524d5747d159c9e" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 41705 } }
I20260806 18:29:53.724620 12122 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:29:53.724747 12122 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:29:53.724867 12122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:53.726080 12122 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2eaa2d8f764513a524d5747d159c9e" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 41705 } }
I20260806 18:29:53.726477 12122 leader_election.cc:302] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e2eaa2d8f764513a524d5747d159c9e; no voters:
I20260806 18:29:53.726902 12122 leader_election.cc:288] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:29:53.727074 12129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:53.727381 12129 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e [term 1 LEADER]: Becoming Leader. State: Replica: 5e2eaa2d8f764513a524d5747d159c9e, State: Running, Role: LEADER
I20260806 18:29:53.727759 12129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2eaa2d8f764513a524d5747d159c9e" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 41705 } }
I20260806 18:29:53.728296 12122 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:29:53.729058 12129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e2eaa2d8f764513a524d5747d159c9e. Latest consensus state: current_term: 1 leader_uuid: "5e2eaa2d8f764513a524d5747d159c9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2eaa2d8f764513a524d5747d159c9e" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 41705 } } }
I20260806 18:29:53.729231 12129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e [sys.catalog]: This master's current role is: LEADER
I20260806 18:29:53.729522 12132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e2eaa2d8f764513a524d5747d159c9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2eaa2d8f764513a524d5747d159c9e" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 41705 } } }
I20260806 18:29:53.729686 12132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e [sys.catalog]: This master's current role is: LEADER
I20260806 18:29:53.729665 12142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:29:53.730835 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:53.715119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43542 (local address 127.7.67.254:41705)
0806 18:29:53.715610 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:53.715635 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:53.715677 (+ 42us) server_negotiation.cc:407] Connection header received
0806 18:29:53.716516 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:53.716541 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:53.716957 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:53.717313 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:53.718144 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:53.719624 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:53.720466 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:53.721186 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:53.721349 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:53.724111 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:53.724117 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:53.724135 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:53.724170 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:53.726404 (+ 2234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:53.726975 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:53.726981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:53.726988 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:53.727140 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:53.727477 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:53.727480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:53.727482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:53.728031 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:53.728217 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:53.728994 (+ 777us) server_negotiation.cc:299] Negotiation successful
0806 18:29:53.729300 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":147,"threads_started":1}
I20260806 18:29:53.733470 12142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:29:53.739141 12142 catalog_manager.cc:1347] Generated new cluster ID: 06e515a864eb458da67eb95b5c707e82
I20260806 18:29:53.739205 12142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:29:53.761144 12142 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:29:53.762096 12142 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:29:53.767459 12142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e2eaa2d8f764513a524d5747d159c9e: Generated new TSK 0
I20260806 18:29:53.768101 12142 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
I20260806 18:29:53.797851 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:41705
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:53.925976 12163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:53.926242 12163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:53.926283 12163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:53.930238 12163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:53.930298 12163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:53.930356 12163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:29:53.935499 12163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:41705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:53.936513 12163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:53.937475 12163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:53.946978 12174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:53.946967 12172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:53.946967 12171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:53.947031 12163 server_base.cc:1034] running on GCE node
I20260806 18:29:53.947628 12163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:53.948297 12163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:53.949465 12163 hybrid_clock.cc:648] HybridClock initialized: now 1786040993949443 us; error 48 us; skew 500 ppm
Aug 06 18:29:53 dist-test-slave-5bn2 krb5kdc[12001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040993, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:53.952203 12163 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:29:53.953081 12163 webserver.cc:460] Webserver started at http://127.7.67.193:33759/ using document root <none> and password file <none>
I20260806 18:29:53.953823 12163 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:53.953918 12163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:53.954165 12163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:53.956317 12163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "56ea0e11847940dfa6ed4ea34b2fa094"
format_stamp: "Formatted at 2026-08-06 18:29:53 on dist-test-slave-5bn2"
server_key: "94583dc2b2eb3b70ea31d70e3a0f1f7a"
server_key_iv: "f8ad4f306ea273be73f3593c284d8b11"
server_key_version: "encryptionkey@0"
I20260806 18:29:53.956854 12163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "56ea0e11847940dfa6ed4ea34b2fa094"
format_stamp: "Formatted at 2026-08-06 18:29:53 on dist-test-slave-5bn2"
server_key: "94583dc2b2eb3b70ea31d70e3a0f1f7a"
server_key_iv: "f8ad4f306ea273be73f3593c284d8b11"
server_key_version: "encryptionkey@0"
I20260806 18:29:53.960572 12163 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:29:53.962987 12183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:53.964221 12163 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:29:53.964349 12163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "56ea0e11847940dfa6ed4ea34b2fa094"
format_stamp: "Formatted at 2026-08-06 18:29:53 on dist-test-slave-5bn2"
server_key: "94583dc2b2eb3b70ea31d70e3a0f1f7a"
server_key_iv: "f8ad4f306ea273be73f3593c284d8b11"
server_key_version: "encryptionkey@0"
I20260806 18:29:53.964449 12163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:53.981266 12163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:53.984345 12163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:53.984540 12163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:53.985145 12163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:53.986003 12163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:53.986054 12163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:53.986109 12163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:53.986135 12163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:53.994905 12163 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:42079
I20260806 18:29:53.994954 12339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:42079 every 8 connection(s)
I20260806 18:29:53.995828 12163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:29:54.005184 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 12163
Aug 06 18:29:54 dist-test-slave-5bn2 krb5kdc[12001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040993, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:54.005312 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:29:54.005642 7439 external_mini_cluster.cc:1428] Setting key be7217e898c1115ac01bfd2410253550
I20260806 18:29:54.009500 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:53.997669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:43215 (local address 127.7.67.254:41705)
0806 18:29:53.997809 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:53.997812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:53.998632 (+ 820us) server_negotiation.cc:407] Connection header received
0806 18:29:54.000002 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:54.000006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:54.000070 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:54.000149 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:54.001351 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:54.001889 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:54.003066 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:54.003309 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:54.003399 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:54.005980 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:54.005984 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:54.005986 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:54.006012 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:54.007546 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:54.008037 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:54.008041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:54.008042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:54.008140 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:54.008430 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:54.008433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:54.008435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:54.008652 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:54.008746 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:54.009237 (+ 491us) server_negotiation.cc:299] Negotiation successful
0806 18:29:54.009370 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:29:54.010643 12344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:53.998015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:41705 (local address 127.7.67.193:43215)
0806 18:29:53.998476 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:53.998520 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:53.999779 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:54.000333 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:54.000340 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:54.000702 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:54.001195 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:54.001209 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:54.002020 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:54.002023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:54.002939 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:54.002950 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:54.003459 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:54.003463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:54.003632 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:54.004175 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:54.004191 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:54.005800 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:54.007661 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:54.007666 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:54.007678 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:54.007935 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:54.008222 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:54.008224 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:54.008225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:54.008356 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:54.008737 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:54.008741 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:54.008985 (+ 244us) client_negotiation.cc:769] Sending connection context
0806 18:29:54.009179 (+ 194us) client_negotiation.cc:240] Negotiation successful
0806 18:29:54.009566 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":114,"threads_started":1}
I20260806 18:29:54.011868 12342 heartbeater.cc:347] Connected to a master server at 127.7.67.254:41705
I20260806 18:29:54.012118 12342 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:54.012650 12342 heartbeater.cc:511] Master 127.7.67.254:41705 requested a full tablet report, sending...
I20260806 18:29:54.014281 12071 ts_manager.cc:194] Registered new tserver with Master: 56ea0e11847940dfa6ed4ea34b2fa094 (127.7.67.193:42079)
I20260806 18:29:54.015457 12071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:43215
WARNING: no policy specified for kudu/127.7.67.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
I20260806 18:29:54.034139 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:41705
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:54.147748 12348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:54.148025 12348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:54.148088 12348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:54.152063 12348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:54.152124 12348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:54.152201 12348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:29:54.157431 12348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:41705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:54.158481 12348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:54.159546 12348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:54.169086 12358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:54.169073 12359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:54.169104 12362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:54.169411 12348 server_base.cc:1034] running on GCE node
I20260806 18:29:54.169832 12348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:54.170521 12348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:54.171695 12348 hybrid_clock.cc:648] HybridClock initialized: now 1786040994171682 us; error 32 us; skew 500 ppm
Aug 06 18:29:54 dist-test-slave-5bn2 krb5kdc[12001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040994, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:54.174451 12348 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:29:54.175345 12348 webserver.cc:460] Webserver started at http://127.7.67.194:46343/ using document root <none> and password file <none>
I20260806 18:29:54.175896 12348 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:54.175964 12348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:54.176162 12348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:54.178237 12348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "e0058405dd054d1bb57b763142926fbf"
format_stamp: "Formatted at 2026-08-06 18:29:54 on dist-test-slave-5bn2"
server_key: "61dc51e28b0d0fccc4c2d752fa19155d"
server_key_iv: "4c4d399129f45b35a9350e60b484bffb"
server_key_version: "encryptionkey@0"
I20260806 18:29:54.178828 12348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "e0058405dd054d1bb57b763142926fbf"
format_stamp: "Formatted at 2026-08-06 18:29:54 on dist-test-slave-5bn2"
server_key: "61dc51e28b0d0fccc4c2d752fa19155d"
server_key_iv: "4c4d399129f45b35a9350e60b484bffb"
server_key_version: "encryptionkey@0"
I20260806 18:29:54.182636 12348 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:29:54.185094 12376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:54.186280 12348 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:29:54.186405 12348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "e0058405dd054d1bb57b763142926fbf"
format_stamp: "Formatted at 2026-08-06 18:29:54 on dist-test-slave-5bn2"
server_key: "61dc51e28b0d0fccc4c2d752fa19155d"
server_key_iv: "4c4d399129f45b35a9350e60b484bffb"
server_key_version: "encryptionkey@0"
I20260806 18:29:54.186502 12348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:54.195114 12348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:54.197939 12348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:54.198114 12348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:54.198684 12348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:54.199621 12348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:54.199671 12348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:54.199725 12348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:54.199754 12348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:54.208261 12348 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:33659
I20260806 18:29:54.208308 12534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:33659 every 8 connection(s)
I20260806 18:29:54.209158 12348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:29:54.210777 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 12348
I20260806 18:29:54.210928 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:29:54.211230 7439 external_mini_cluster.cc:1428] Setting key 4bf67bc8a12725e6eee8fd78d0333f77
Aug 06 18:29:54 dist-test-slave-5bn2 krb5kdc[12001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040994, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.195@KRBTEST.COM; defaulting to no policy
I20260806 18:29:54.224211 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:54.210994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:37703 (local address 127.7.67.254:41705)
0806 18:29:54.211134 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:54.211140 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:54.212251 (+ 1111us) server_negotiation.cc:407] Connection header received
0806 18:29:54.213900 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:54.213904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:54.213957 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:54.214031 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:54.215541 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:54.216295 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:54.217424 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:54.217788 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:54.217900 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:54.220695 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:54.220698 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:54.220699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:54.220726 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:54.222076 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:54.222559 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:54.222561 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:54.222562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:54.222637 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:54.223003 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:54.223006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:54.223008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:54.223200 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:54.223294 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:54.223837 (+ 543us) server_negotiation.cc:299] Negotiation successful
0806 18:29:54.224044 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.195@KRBTEST.COM" created.
I20260806 18:29:54.225081 12538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:54.211582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:41705 (local address 127.7.67.194:37703)
0806 18:29:54.212090 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:54.212143 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:54.213658 (+ 1515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:54.214211 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:54.214219 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:54.214692 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:54.215388 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:54.215405 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:54.216414 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:54.216417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:54.217294 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:54.217303 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:54.217975 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:54.217980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:54.218163 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:54.218715 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:54.218738 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:54.220515 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:54.222188 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:54.222195 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:54.222211 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:54.222476 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:54.222702 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:54.222704 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:54.222705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:54.222896 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:54.223307 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:54.223314 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:54.223585 (+ 271us) client_negotiation.cc:769] Sending connection context
0806 18:29:54.223753 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 18:29:54.224109 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":110,"threads_started":1}
I20260806 18:29:54.226346 12536 heartbeater.cc:347] Connected to a master server at 127.7.67.254:41705
I20260806 18:29:54.226626 12536 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:54.227171 12536 heartbeater.cc:511] Master 127.7.67.254:41705 requested a full tablet report, sending...
I20260806 18:29:54.228225 12071 ts_manager.cc:194] Registered new tserver with Master: e0058405dd054d1bb57b763142926fbf (127.7.67.194:33659)
I20260806 18:29:54.228904 12071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:37703
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
I20260806 18:29:54.239720 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:41705
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:54.352766 12541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:54.353044 12541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:54.353087 12541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:54.357087 12541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:54.357148 12541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:54.357206 12541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:29:54.362423 12541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:41705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:54.363577 12541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:54.364616 12541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:54.374536 12549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:54.374544 12551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:54.374800 12541 server_base.cc:1034] running on GCE node
W20260806 18:29:54.374923 12547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:54.375345 12541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:54.376008 12541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:54.377166 12541 hybrid_clock.cc:648] HybridClock initialized: now 1786040994377135 us; error 44 us; skew 500 ppm
Aug 06 18:29:54 dist-test-slave-5bn2 krb5kdc[12001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040994, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:54.379873 12541 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:29:54.380746 12541 webserver.cc:460] Webserver started at http://127.7.67.195:34143/ using document root <none> and password file <none>
I20260806 18:29:54.381361 12541 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:54.381434 12541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:54.381675 12541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:54.383981 12541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "cff10fdaec874d42b84dd72e2a4b7d62"
format_stamp: "Formatted at 2026-08-06 18:29:54 on dist-test-slave-5bn2"
server_key: "2daeec1bd783e295aff59f8eae47dc46"
server_key_iv: "b421f5d5bd07d23a2d9fdd270ac3de01"
server_key_version: "encryptionkey@0"
I20260806 18:29:54.384549 12541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "cff10fdaec874d42b84dd72e2a4b7d62"
format_stamp: "Formatted at 2026-08-06 18:29:54 on dist-test-slave-5bn2"
server_key: "2daeec1bd783e295aff59f8eae47dc46"
server_key_iv: "b421f5d5bd07d23a2d9fdd270ac3de01"
server_key_version: "encryptionkey@0"
I20260806 18:29:54.388276 12541 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:29:54.390723 12564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:54.392019 12541 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:29:54.392143 12541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "cff10fdaec874d42b84dd72e2a4b7d62"
format_stamp: "Formatted at 2026-08-06 18:29:54 on dist-test-slave-5bn2"
server_key: "2daeec1bd783e295aff59f8eae47dc46"
server_key_iv: "b421f5d5bd07d23a2d9fdd270ac3de01"
server_key_version: "encryptionkey@0"
I20260806 18:29:54.392241 12541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:54.420011 12541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:54.422827 12541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:54.423039 12541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:54.423656 12541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:54.424515 12541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:54.424563 12541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:54.424620 12541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:54.424646 12541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:54.433317 12541 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:37057
I20260806 18:29:54.433349 12725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:37057 every 8 connection(s)
I20260806 18:29:54.434206 12541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:29:54.436205 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 12541
I20260806 18:29:54.436313 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:29:54.436591 7439 external_mini_cluster.cc:1428] Setting key 0784c631fda9c8bf85dfb5a4846df66c
Aug 06 18:29:54 dist-test-slave-5bn2 krb5kdc[12001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040994, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:54.447345 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:54.436039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:34861 (local address 127.7.67.254:41705)
0806 18:29:54.436189 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:54.436194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:54.437124 (+ 930us) server_negotiation.cc:407] Connection header received
0806 18:29:54.438689 (+ 1565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:54.438692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:54.438748 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:54.438818 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:54.440013 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:54.440536 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:54.441658 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:54.441878 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:54.441964 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:54.444412 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:54.444415 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:54.444417 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:54.444442 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:54.445590 (+ 1148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:54.446056 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:54.446059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:54.446060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:54.446134 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:54.446400 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:54.446401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:54.446403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:54.446563 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:54.446631 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:54.447117 (+ 486us) server_negotiation.cc:299] Negotiation successful
0806 18:29:54.447215 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:29:54.448146 12732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:54.436459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:41705 (local address 127.7.67.195:34861)
0806 18:29:54.436980 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:54.437027 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:54.438468 (+ 1441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:54.439016 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:54.439023 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:54.439381 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:54.439871 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:54.439883 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:54.440658 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:54.440661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:54.441542 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:54.441552 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:54.442019 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:54.442024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:54.442197 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:54.442746 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:54.442763 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:54.444247 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:54.445689 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:54.445694 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:54.445707 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:54.445974 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:54.446197 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:54.446200 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:54.446201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:54.446335 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:54.446633 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:54.446637 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:54.446908 (+ 271us) client_negotiation.cc:769] Sending connection context
0806 18:29:54.447070 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 18:29:54.447329 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":126,"threads_started":1}
I20260806 18:29:54.449271 12726 heartbeater.cc:347] Connected to a master server at 127.7.67.254:41705
I20260806 18:29:54.449540 12726 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:54.450145 12726 heartbeater.cc:511] Master 127.7.67.254:41705 requested a full tablet report, sending...
I20260806 18:29:54.451218 12071 ts_manager.cc:194] Registered new tserver with Master: cff10fdaec874d42b84dd72e2a4b7d62 (127.7.67.195:37057)
I20260806 18:29:54.451781 12071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:34861
I20260806 18:29:54.462019 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:29:54 dist-test-slave-5bn2 krb5kdc[12001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040994, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 18:29:54 dist-test-slave-5bn2 krb5kdc[12001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040994, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:54.487478 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:54.479802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43546 (local address 127.7.67.254:41705)
0806 18:29:54.479966 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:54.479972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:54.480083 (+ 111us) server_negotiation.cc:407] Connection header received
0806 18:29:54.480206 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:54.480209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:54.480254 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:54.480323 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:54.480955 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:54.481470 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:54.482281 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:54.482516 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:54.482593 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:54.484826 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:54.484830 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:54.484832 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:54.484857 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:54.485986 (+ 1129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:54.486347 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:54.486349 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:54.486350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:54.486417 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:54.486610 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:54.486612 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:54.486613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:54.486817 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:54.487026 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:54.487214 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 18:29:54.487315 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 18:29:54.487860 12071 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43546
I20260806 18:29:54.488822 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 12163
I20260806 18:29:54.497840 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 12348
I20260806 18:29:54.509605 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 12541
I20260806 18:29:54.519547 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 12019
2026-08-06T18:29:54Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2235 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:29:54 dist-test-slave-5bn2 krb5kdc[12747](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:29:54 dist-test-slave-5bn2 krb5kdc[12747](info): set up 2 sockets
Aug 06 18:29:54 dist-test-slave-5bn2 krb5kdc[12747](info): commencing operation
krb5kdc: starting...
W20260806 18:29:55.632417 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:29:55 dist-test-slave-5bn2 krb5kdc[12747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:29:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:29:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:29:55.754896 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:39077
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:39077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:55.868669 12767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:55.868961 12767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:55.869021 12767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:55.873051 12767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:29:55.873121 12767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:55.873148 12767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:55.873168 12767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:29:55.873189 12767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:29:55.878523 12767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:39077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:39077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:55.879678 12767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:55.880767 12767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:55.890549 12774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:55.890547 12775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:55.890540 12777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:55.890620 12767 server_base.cc:1034] running on GCE node
I20260806 18:29:55.891448 12767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:55.892598 12767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:55.893759 12767 hybrid_clock.cc:648] HybridClock initialized: now 1786040995893743 us; error 28 us; skew 500 ppm
Aug 06 18:29:55 dist-test-slave-5bn2 krb5kdc[12747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040995, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:55.896526 12767 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:29:55.897457 12767 webserver.cc:460] Webserver started at http://127.7.67.254:44311/ using document root <none> and password file <none>
I20260806 18:29:55.898038 12767 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:55.898105 12767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:55.898319 12767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:55.900475 12767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "f2fd307e3fb74e2ca321092a50baaeb4"
format_stamp: "Formatted at 2026-08-06 18:29:55 on dist-test-slave-5bn2"
server_key: "5631ded2b30c904775f7a99b8eb8dd22"
server_key_iv: "c5d7fdf4485ae3fc70e69d0e1ebf5761"
server_key_version: "encryptionkey@0"
I20260806 18:29:55.901127 12767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "f2fd307e3fb74e2ca321092a50baaeb4"
format_stamp: "Formatted at 2026-08-06 18:29:55 on dist-test-slave-5bn2"
server_key: "5631ded2b30c904775f7a99b8eb8dd22"
server_key_iv: "c5d7fdf4485ae3fc70e69d0e1ebf5761"
server_key_version: "encryptionkey@0"
I20260806 18:29:55.904990 12767 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260806 18:29:55.907500 12785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:55.908680 12767 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:29:55.908797 12767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "f2fd307e3fb74e2ca321092a50baaeb4"
format_stamp: "Formatted at 2026-08-06 18:29:55 on dist-test-slave-5bn2"
server_key: "5631ded2b30c904775f7a99b8eb8dd22"
server_key_iv: "c5d7fdf4485ae3fc70e69d0e1ebf5761"
server_key_version: "encryptionkey@0"
I20260806 18:29:55.908917 12767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:55.943853 12767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:55.947096 12767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:55.947286 12767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:55.954511 12767 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:39077
I20260806 18:29:55.954532 12858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:39077 every 8 connection(s)
I20260806 18:29:55.955616 12767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:29:55.958503 12859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:55.961414 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 12767
I20260806 18:29:55.961524 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:29:55.961820 7439 external_mini_cluster.cc:1428] Setting key 7c1bf4f89926ba6d5fdd83b1a492f708
I20260806 18:29:55.964867 12859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4: Bootstrap starting.
I20260806 18:29:55.967618 12859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:55.968381 12859 log.cc:824] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4: Log is configured to *not* fsync() on all Append() calls
I20260806 18:29:55.970947 12859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4: No bootstrap required, opened a new log
Aug 06 18:29:55 dist-test-slave-5bn2 krb5kdc[12747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:55.974056 12859 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2fd307e3fb74e2ca321092a50baaeb4" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39077 } }
I20260806 18:29:55.974293 12859 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:55.974346 12859 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2fd307e3fb74e2ca321092a50baaeb4, State: Initialized, Role: FOLLOWER
I20260806 18:29:55.974812 12859 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2fd307e3fb74e2ca321092a50baaeb4" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39077 } }
I20260806 18:29:55.974977 12859 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:29:55.975044 12859 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:29:55.975145 12859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:55.976063 12859 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2fd307e3fb74e2ca321092a50baaeb4" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39077 } }
I20260806 18:29:55.976460 12859 leader_election.cc:302] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2fd307e3fb74e2ca321092a50baaeb4; no voters:
I20260806 18:29:55.976800 12859 leader_election.cc:288] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:29:55.976902 12865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:55.977139 12865 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4 [term 1 LEADER]: Becoming Leader. State: Replica: f2fd307e3fb74e2ca321092a50baaeb4, State: Running, Role: LEADER
I20260806 18:29:55.977296 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:55.963351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48530 (local address 127.7.67.254:39077)
0806 18:29:55.963791 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:55.963812 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:55.963864 (+ 52us) server_negotiation.cc:407] Connection header received
0806 18:29:55.964734 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:55.964760 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:55.965184 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:55.965487 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:55.966493 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:55.967608 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:55.968504 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:55.969133 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:55.969289 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:55.972004 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:55.972010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:55.972024 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:55.972053 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:55.974141 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:55.974612 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:55.974618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:55.974622 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:55.974740 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:55.975088 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:55.975092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:55.975094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:55.975542 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:55.975738 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:55.976042 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 18:29:55.976337 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":127,"threads_started":1}
I20260806 18:29:55.977541 12865 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2fd307e3fb74e2ca321092a50baaeb4" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39077 } }
I20260806 18:29:55.978130 12859 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:29:55.979259 12868 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2fd307e3fb74e2ca321092a50baaeb4. Latest consensus state: current_term: 1 leader_uuid: "f2fd307e3fb74e2ca321092a50baaeb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2fd307e3fb74e2ca321092a50baaeb4" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39077 } } }
I20260806 18:29:55.979645 12868 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4 [sys.catalog]: This master's current role is: LEADER
I20260806 18:29:55.979547 12866 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2fd307e3fb74e2ca321092a50baaeb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2fd307e3fb74e2ca321092a50baaeb4" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39077 } } }
I20260806 18:29:55.979990 12866 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4 [sys.catalog]: This master's current role is: LEADER
W20260806 18:29:55.982897 12888 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:29:55.982975 12888 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:29:55.983079 12883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:29:55.984077 12883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:29:55.990134 12883 catalog_manager.cc:1347] Generated new cluster ID: 319e3558e2d94d88b520d17db79b0dc6
I20260806 18:29:55.990221 12883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:29:55.999392 12883 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:29:56.000365 12883 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:29:56.011826 12883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2fd307e3fb74e2ca321092a50baaeb4: Generated new TSK 0
I20260806 18:29:56.012624 12883 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
I20260806 18:29:56.049418 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39077
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:56.170004 12896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:56.170346 12896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:56.170397 12896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:56.174606 12896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:56.174703 12896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:56.174768 12896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:29:56.180765 12896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:56.181968 12896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:56.183120 12896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:56.193475 12902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:56.193475 12905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:56.193547 12903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:56.193660 12896 server_base.cc:1034] running on GCE node
I20260806 18:29:56.194218 12896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:56.194937 12896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:56.196125 12896 hybrid_clock.cc:648] HybridClock initialized: now 1786040996196099 us; error 49 us; skew 500 ppm
Aug 06 18:29:56 dist-test-slave-5bn2 krb5kdc[12747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040996, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:56.199126 12896 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:29:56.200166 12896 webserver.cc:460] Webserver started at http://127.7.67.193:45963/ using document root <none> and password file <none>
I20260806 18:29:56.200752 12896 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:56.200821 12896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:56.201056 12896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:56.203284 12896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "62915bae58314b358f8e0b96d555e873"
format_stamp: "Formatted at 2026-08-06 18:29:56 on dist-test-slave-5bn2"
server_key: "253545a2493beee97f0404cd3350422e"
server_key_iv: "be0acc019d549cedbd012a5a99ca7a06"
server_key_version: "encryptionkey@0"
I20260806 18:29:56.203891 12896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "62915bae58314b358f8e0b96d555e873"
format_stamp: "Formatted at 2026-08-06 18:29:56 on dist-test-slave-5bn2"
server_key: "253545a2493beee97f0404cd3350422e"
server_key_iv: "be0acc019d549cedbd012a5a99ca7a06"
server_key_version: "encryptionkey@0"
I20260806 18:29:56.207981 12896 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:29:56.210774 12914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:56.212160 12896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:29:56.212283 12896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "62915bae58314b358f8e0b96d555e873"
format_stamp: "Formatted at 2026-08-06 18:29:56 on dist-test-slave-5bn2"
server_key: "253545a2493beee97f0404cd3350422e"
server_key_iv: "be0acc019d549cedbd012a5a99ca7a06"
server_key_version: "encryptionkey@0"
I20260806 18:29:56.212394 12896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:56.230342 12896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:56.233697 12896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:56.233891 12896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:56.234514 12896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:56.235433 12896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:56.235484 12896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:56.235538 12896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:56.235579 12896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:56.244146 12896 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:42829
I20260806 18:29:56.244200 13078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:42829 every 8 connection(s)
I20260806 18:29:56.245069 12896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:29:56.247406 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 12896
I20260806 18:29:56.247524 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:29:56.247869 7439 external_mini_cluster.cc:1428] Setting key 0f1f6f886311c4c3552e2ee7197a6804
Aug 06 18:29:56 dist-test-slave-5bn2 krb5kdc[12747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040996, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.194@KRBTEST.COM; defaulting to no policy
I20260806 18:29:56.259761 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:56.246907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:35467 (local address 127.7.67.254:39077)
0806 18:29:56.247053 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:56.247057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:56.248047 (+ 990us) server_negotiation.cc:407] Connection header received
0806 18:29:56.249640 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:56.249643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:56.249689 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:56.249750 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:56.251340 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:56.252130 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:56.253352 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:56.253594 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:56.253684 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:56.256233 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:56.256236 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:56.256238 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:56.256263 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:56.257627 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:56.258136 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:56.258140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:56.258142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:56.258234 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:56.258586 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:56.258588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:56.258589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:56.258769 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:56.258878 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:56.259427 (+ 549us) server_negotiation.cc:299] Negotiation successful
0806 18:29:56.259584 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:29:56.260553 13082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:56.247309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39077 (local address 127.7.67.193:35467)
0806 18:29:56.247859 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:56.247929 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:56.249420 (+ 1491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:56.249966 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:56.249976 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:56.250477 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:56.251193 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:56.251213 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:56.252270 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:56.252273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:56.253225 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:56.253235 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:56.253746 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:56.253750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:56.253919 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:56.254537 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:56.254556 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:56.256064 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:56.257739 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:56.257745 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:56.257763 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:56.258022 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:56.258317 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:56.258320 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:56.258322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:56.258476 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:56.258893 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:56.258900 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:56.259183 (+ 283us) client_negotiation.cc:769] Sending connection context
0806 18:29:56.259347 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 18:29:56.259699 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":132,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.194@KRBTEST.COM" created.
I20260806 18:29:56.261813 13079 heartbeater.cc:347] Connected to a master server at 127.7.67.254:39077
I20260806 18:29:56.262070 13079 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:56.262607 13079 heartbeater.cc:511] Master 127.7.67.254:39077 requested a full tablet report, sending...
I20260806 18:29:56.264266 12808 ts_manager.cc:194] Registered new tserver with Master: 62915bae58314b358f8e0b96d555e873 (127.7.67.193:42829)
I20260806 18:29:56.265440 12808 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:35467
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
I20260806 18:29:56.276237 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39077
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:56.392496 13085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:56.392787 13085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:56.392839 13085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:56.396909 13085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:56.396981 13085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:56.397042 13085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:29:56.402779 13085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:56.403906 13085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:56.405001 13085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:56.415131 13090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:56.415211 13085 server_base.cc:1034] running on GCE node
W20260806 18:29:56.415228 13091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:56.415144 13094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:56.415833 13085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:56.416543 13085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:56.417721 13085 hybrid_clock.cc:648] HybridClock initialized: now 1786040996417681 us; error 50 us; skew 500 ppm
Aug 06 18:29:56 dist-test-slave-5bn2 krb5kdc[12747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040996, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:56.420666 13085 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:29:56.421571 13085 webserver.cc:460] Webserver started at http://127.7.67.194:35871/ using document root <none> and password file <none>
I20260806 18:29:56.422130 13085 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:56.422204 13085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:56.422408 13085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:56.424554 13085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "5c35edeff99040fba35ca731331f5122"
format_stamp: "Formatted at 2026-08-06 18:29:56 on dist-test-slave-5bn2"
server_key: "bf549ac77e39e6d5e673427b4e043f85"
server_key_iv: "d8e690ac29445d8dfa336e4c6b59c1e1"
server_key_version: "encryptionkey@0"
I20260806 18:29:56.425123 13085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "5c35edeff99040fba35ca731331f5122"
format_stamp: "Formatted at 2026-08-06 18:29:56 on dist-test-slave-5bn2"
server_key: "bf549ac77e39e6d5e673427b4e043f85"
server_key_iv: "d8e690ac29445d8dfa336e4c6b59c1e1"
server_key_version: "encryptionkey@0"
I20260806 18:29:56.428974 13085 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:29:56.431547 13103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:56.432778 13085 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:29:56.432893 13085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "5c35edeff99040fba35ca731331f5122"
format_stamp: "Formatted at 2026-08-06 18:29:56 on dist-test-slave-5bn2"
server_key: "bf549ac77e39e6d5e673427b4e043f85"
server_key_iv: "d8e690ac29445d8dfa336e4c6b59c1e1"
server_key_version: "encryptionkey@0"
I20260806 18:29:56.433003 13085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:56.449615 13085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:56.452872 13085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:56.453070 13085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:56.453692 13085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:56.454550 13085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:56.454599 13085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:56.454654 13085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:56.454682 13085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:56.463189 13085 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:41043
I20260806 18:29:56.463223 13257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:41043 every 8 connection(s)
I20260806 18:29:56.464092 13085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
Aug 06 18:29:56 dist-test-slave-5bn2 krb5kdc[12747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040996, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:56.473840 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 13085
I20260806 18:29:56.473958 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:29:56.474228 7439 external_mini_cluster.cc:1428] Setting key 957eb0ed5413ccffcc596851642e15af
I20260806 18:29:56.477435 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:56.465902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:49531 (local address 127.7.67.254:39077)
0806 18:29:56.466073 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:56.466078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:56.466965 (+ 887us) server_negotiation.cc:407] Connection header received
0806 18:29:56.468324 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:56.468327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:56.468407 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:56.468483 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:56.469659 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:56.470204 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:56.471347 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:56.471558 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:56.471633 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:56.473953 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:56.473957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:56.473961 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:56.473990 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:56.475444 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:56.475984 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:56.475988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:56.475990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:56.476073 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:56.476383 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:56.476386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:56.476388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:56.476571 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:56.476656 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:56.477148 (+ 492us) server_negotiation.cc:299] Negotiation successful
0806 18:29:56.477271 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 18:29:56.478529 13260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:56.466308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39077 (local address 127.7.67.194:49531)
0806 18:29:56.466780 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:56.466823 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:56.468124 (+ 1301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:56.468644 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:56.468650 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:56.469005 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:56.469520 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:56.469535 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:56.470294 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:56.470297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:56.471247 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:56.471257 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:56.471676 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:56.471679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:56.471815 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:56.472316 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:56.472332 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:56.473785 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:56.475566 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:56.475572 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:56.475588 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:56.475887 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:56.476154 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:56.476158 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:56.476160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:56.476294 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:56.476669 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:56.476676 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:56.476920 (+ 244us) client_negotiation.cc:769] Sending connection context
0806 18:29:56.477087 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 18:29:56.477428 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":121,"threads_started":1}
I20260806 18:29:56.479827 13258 heartbeater.cc:347] Connected to a master server at 127.7.67.254:39077
I20260806 18:29:56.480077 13258 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:56.480657 13258 heartbeater.cc:511] Master 127.7.67.254:39077 requested a full tablet report, sending...
I20260806 18:29:56.481776 12808 ts_manager.cc:194] Registered new tserver with Master: 5c35edeff99040fba35ca731331f5122 (127.7.67.194:41043)
I20260806 18:29:56.482537 12808 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:49531
WARNING: no policy specified for kudu/127.7.67.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
I20260806 18:29:56.501997 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39077
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:56.618057 13264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:56.618340 13264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:56.618391 13264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:56.622623 13264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:56.622706 13264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:56.622774 13264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:29:56.628510 13264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:56.629622 13264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:56.630684 13264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:56.640635 13273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:56.642270 13272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:56.642654 13264 server_base.cc:1034] running on GCE node
W20260806 18:29:56.642727 13276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:56.643169 13264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:56.643829 13264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:56.644989 13264 hybrid_clock.cc:648] HybridClock initialized: now 1786040996644968 us; error 34 us; skew 500 ppm
Aug 06 18:29:56 dist-test-slave-5bn2 krb5kdc[12747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040996, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:56.647773 13264 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:29:56.648725 13264 webserver.cc:460] Webserver started at http://127.7.67.195:45377/ using document root <none> and password file <none>
I20260806 18:29:56.649315 13264 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:56.649380 13264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:56.649591 13264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:56.651813 13264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "975d0348b7b24116988281c630020771"
format_stamp: "Formatted at 2026-08-06 18:29:56 on dist-test-slave-5bn2"
server_key: "45442f26f0ada1ca2bd0ffe71ca90d70"
server_key_iv: "53516b2a7e47578ac75e73c50f4a89f6"
server_key_version: "encryptionkey@0"
I20260806 18:29:56.652393 13264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "975d0348b7b24116988281c630020771"
format_stamp: "Formatted at 2026-08-06 18:29:56 on dist-test-slave-5bn2"
server_key: "45442f26f0ada1ca2bd0ffe71ca90d70"
server_key_iv: "53516b2a7e47578ac75e73c50f4a89f6"
server_key_version: "encryptionkey@0"
I20260806 18:29:56.656284 13264 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:29:56.658727 13284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:56.659981 13264 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:29:56.660108 13264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "975d0348b7b24116988281c630020771"
format_stamp: "Formatted at 2026-08-06 18:29:56 on dist-test-slave-5bn2"
server_key: "45442f26f0ada1ca2bd0ffe71ca90d70"
server_key_iv: "53516b2a7e47578ac75e73c50f4a89f6"
server_key_version: "encryptionkey@0"
I20260806 18:29:56.660212 13264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:56.672923 13264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:56.675743 13264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:56.675923 13264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:56.676544 13264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:56.677407 13264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:56.677456 13264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:56.677513 13264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:56.677543 13264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:56.686287 13264 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:42359
I20260806 18:29:56.686312 13442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:42359 every 8 connection(s)
I20260806 18:29:56.687286 13264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:29:56.689483 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 13264
I20260806 18:29:56.689620 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:29:56.689888 7439 external_mini_cluster.cc:1428] Setting key 6f6e050cda878be001fad5cd3683275a
Aug 06 18:29:56 dist-test-slave-5bn2 krb5kdc[12747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040996, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:56.700690 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:56.689181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:34501 (local address 127.7.67.254:39077)
0806 18:29:56.689336 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:56.689341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:56.690289 (+ 948us) server_negotiation.cc:407] Connection header received
0806 18:29:56.691764 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:56.691767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:56.691824 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:56.691896 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:56.693081 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:56.693628 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:56.694763 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:56.695042 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:56.695135 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:56.697474 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:56.697478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:56.697481 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:56.697509 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:56.698638 (+ 1129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:56.699180 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:56.699183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:56.699185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:56.699270 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:56.699595 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:56.699598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:56.699600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:56.699783 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:56.699870 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:56.700378 (+ 508us) server_negotiation.cc:299] Negotiation successful
0806 18:29:56.700505 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:29:56.701478 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:56.689554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39077 (local address 127.7.67.195:34501)
0806 18:29:56.690114 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:56.690171 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:56.691554 (+ 1383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:56.692077 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:56.692084 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:56.692442 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:56.692939 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:56.692953 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:56.693751 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:56.693754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:56.694643 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:56.694653 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:56.695192 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:56.695196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:56.695359 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:56.695874 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:56.695890 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:56.697312 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:56.698746 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:56.698752 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:56.698767 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:56.699084 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:56.699353 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:56.699356 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:56.699358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:56.699500 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:56.699885 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:56.699892 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:56.700144 (+ 252us) client_negotiation.cc:769] Sending connection context
0806 18:29:56.700315 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 18:29:56.700647 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":119,"threads_started":1}
I20260806 18:29:56.702631 13443 heartbeater.cc:347] Connected to a master server at 127.7.67.254:39077
I20260806 18:29:56.702913 13443 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:56.703523 13443 heartbeater.cc:511] Master 127.7.67.254:39077 requested a full tablet report, sending...
I20260806 18:29:56.704509 12808 ts_manager.cc:194] Registered new tserver with Master: 975d0348b7b24116988281c630020771 (127.7.67.195:42359)
I20260806 18:29:56.705039 12808 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:34501
I20260806 18:29:56.715494 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:29:56 dist-test-slave-5bn2 krb5kdc[12747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:29:56 dist-test-slave-5bn2 krb5kdc[12747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.193@KRBTEST.COM
I20260806 18:29:56.743595 13452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:56.733429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50172 (local address 127.7.67.193:42829)
0806 18:29:56.733737 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:56.733753 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:56.733779 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:29:56.733833 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:56.733838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:56.734042 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:56.734164 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:56.734925 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:56.736126 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:56.737354 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:56.737925 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:56.738021 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:56.740458 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:56.740468 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:56.740483 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:56.740510 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:56.741912 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:56.742299 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:56.742304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:56.742305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:56.742422 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:56.742618 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:56.742619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:56.742620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:56.742930 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:56.743104 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:56.743252 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 18:29:56.743430 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":105,"threads_started":1}
Aug 06 18:29:56 dist-test-slave-5bn2 krb5kdc[12747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:56.753993 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:56.746594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34086 (local address 127.7.67.254:39077)
0806 18:29:56.746736 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:56.746740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:56.746767 (+ 27us) server_negotiation.cc:407] Connection header received
0806 18:29:56.746921 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:56.746923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:56.746967 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:56.747040 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:56.747588 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:56.748094 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:56.748841 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:56.749044 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:56.749108 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:56.751294 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:56.751297 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:56.751299 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:56.751325 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:56.752616 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:56.752980 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:56.752982 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:56.752983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:56.753059 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:56.753254 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:56.753255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:56.753257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:56.753459 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:56.753562 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:56.753731 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 18:29:56.753824 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 18:29:56.754454 12808 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34086
I20260806 18:29:56.755090 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 12896
I20260806 18:29:56.767050 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 13085
I20260806 18:29:56.786370 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 13264
I20260806 18:29:56.798025 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 12767
2026-08-06T18:29:56Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2276 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:29:56 dist-test-slave-5bn2 krb5kdc[13463](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:29:56 dist-test-slave-5bn2 krb5kdc[13463](info): set up 2 sockets
Aug 06 18:29:56 dist-test-slave-5bn2 krb5kdc[13463](info): commencing operation
krb5kdc: starting...
W20260806 18:29:57.925884 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:29:58 dist-test-slave-5bn2 krb5kdc[13463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:29:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:29:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:29:58.051486 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:39289
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:36927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:39289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:58.165469 13483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:58.165751 13483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:58.165810 13483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:58.169853 13483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:29:58.169922 13483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:58.169950 13483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:58.169970 13483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:29:58.169989 13483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:29:58.175348 13483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:36927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:39289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:39289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:58.176497 13483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:58.177542 13483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:58.185863 13491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:58.185858 13498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:58.185863 13492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:58.186215 13483 server_base.cc:1034] running on GCE node
I20260806 18:29:58.186831 13483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:58.188012 13483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:58.189196 13483 hybrid_clock.cc:648] HybridClock initialized: now 1786040998189170 us; error 37 us; skew 500 ppm
Aug 06 18:29:58 dist-test-slave-5bn2 krb5kdc[13463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040998, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:58.192106 13483 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:29:58.192884 13483 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:58.192950 13483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:58.193143 13483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:58.195266 13483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "74796f2c1a3f4946b0edc51a10b1a5f5"
format_stamp: "Formatted at 2026-08-06 18:29:58 on dist-test-slave-5bn2"
server_key: "9f23855f33be5a5b21628b06075a665c"
server_key_iv: "f340d6dbec2376bd6fd1259fc1f036ce"
server_key_version: "encryptionkey@0"
I20260806 18:29:58.195844 13483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "74796f2c1a3f4946b0edc51a10b1a5f5"
format_stamp: "Formatted at 2026-08-06 18:29:58 on dist-test-slave-5bn2"
server_key: "9f23855f33be5a5b21628b06075a665c"
server_key_iv: "f340d6dbec2376bd6fd1259fc1f036ce"
server_key_version: "encryptionkey@0"
I20260806 18:29:58.199586 13483 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:29:58.202011 13505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:58.203281 13483 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260806 18:29:58.203408 13483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "74796f2c1a3f4946b0edc51a10b1a5f5"
format_stamp: "Formatted at 2026-08-06 18:29:58 on dist-test-slave-5bn2"
server_key: "9f23855f33be5a5b21628b06075a665c"
server_key_iv: "f340d6dbec2376bd6fd1259fc1f036ce"
server_key_version: "encryptionkey@0"
I20260806 18:29:58.203516 13483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:58.220592 13483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:58.223762 13483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:58.223951 13483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:58.230928 13483 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:39289
I20260806 18:29:58.231163 13582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:39289 every 8 connection(s)
I20260806 18:29:58.231621 13483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:29:58.234372 13584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:58.238914 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 13483
I20260806 18:29:58.239049 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:29:58.239316 7439 external_mini_cluster.cc:1428] Setting key b509af75199470710b48a12c2d704c76
I20260806 18:29:58.240162 13584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5: Bootstrap starting.
I20260806 18:29:58.242565 13584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:58.243536 13584 log.cc:824] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5: Log is configured to *not* fsync() on all Append() calls
I20260806 18:29:58.245687 13584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5: No bootstrap required, opened a new log
Aug 06 18:29:58 dist-test-slave-5bn2 krb5kdc[13463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:58.250089 13584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74796f2c1a3f4946b0edc51a10b1a5f5" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39289 } }
I20260806 18:29:58.250396 13584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:58.250454 13584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74796f2c1a3f4946b0edc51a10b1a5f5, State: Initialized, Role: FOLLOWER
I20260806 18:29:58.250991 13584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74796f2c1a3f4946b0edc51a10b1a5f5" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39289 } }
I20260806 18:29:58.251154 13584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:29:58.251227 13584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:29:58.251324 13584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:58.252419 13584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74796f2c1a3f4946b0edc51a10b1a5f5" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39289 } }
I20260806 18:29:58.252792 13584 leader_election.cc:302] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74796f2c1a3f4946b0edc51a10b1a5f5; no voters:
I20260806 18:29:58.253176 13584 leader_election.cc:288] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:29:58.253268 13590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:58.253489 13590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5 [term 1 LEADER]: Becoming Leader. State: Replica: 74796f2c1a3f4946b0edc51a10b1a5f5, State: Running, Role: LEADER
I20260806 18:29:58.253897 13590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74796f2c1a3f4946b0edc51a10b1a5f5" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39289 } }
I20260806 18:29:58.254199 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:58.240716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41384 (local address 127.7.67.254:39289)
0806 18:29:58.241127 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:58.241143 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:58.241180 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:29:58.241843 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:58.241861 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:58.242192 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:58.242495 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:58.243445 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:58.244628 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:58.245743 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:58.246225 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:58.246353 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:58.249055 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:58.249062 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:58.249079 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:58.249112 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:58.251048 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:58.251529 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:58.251535 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:58.251541 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:58.251683 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:58.251913 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:58.251916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:58.251917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:58.252449 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:58.252666 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:58.252906 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 18:29:58.253218 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":109,"threads_started":1}
I20260806 18:29:58.254468 13584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:29:58.255481 13591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74796f2c1a3f4946b0edc51a10b1a5f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74796f2c1a3f4946b0edc51a10b1a5f5" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39289 } } }
I20260806 18:29:58.255607 13591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:29:58.255857 13591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74796f2c1a3f4946b0edc51a10b1a5f5. Latest consensus state: current_term: 1 leader_uuid: "74796f2c1a3f4946b0edc51a10b1a5f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74796f2c1a3f4946b0edc51a10b1a5f5" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39289 } } }
I20260806 18:29:58.255934 13591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:29:58.256896 13602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:29:58.259673 13602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:29:58.264873 13602 catalog_manager.cc:1347] Generated new cluster ID: 61e217f037c642dfb8b6b5b0a5022168
I20260806 18:29:58.264945 13602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:29:58.274348 13602 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:29:58.275691 13602 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:29:58.283800 13602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5: Generated new TSK 0
I20260806 18:29:58.284508 13602 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
I20260806 18:29:58.323745 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39289
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:36927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:58.439231 13622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:58.439512 13622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:58.439555 13622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:58.443583 13622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:58.443645 13622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:58.443706 13622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:29:58.448890 13622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:36927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:58.449955 13622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:58.451107 13622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:58.460728 13633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:58.460729 13631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:58.460729 13629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:58.460954 13622 server_base.cc:1034] running on GCE node
I20260806 18:29:58.461445 13622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:58.462059 13622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:58.463218 13622 hybrid_clock.cc:648] HybridClock initialized: now 1786040998463186 us; error 42 us; skew 500 ppm
Aug 06 18:29:58 dist-test-slave-5bn2 krb5kdc[13463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040998, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:58.466090 13622 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:29:58.466892 13622 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:58.466972 13622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:58.467187 13622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:58.469303 13622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "c5aa7371d8234fd884a6daea95706f94"
format_stamp: "Formatted at 2026-08-06 18:29:58 on dist-test-slave-5bn2"
server_key: "af7964c20af7ff5edce2b0a3313905e4"
server_key_iv: "699ee2504ddfb961d7e4af9cb395ff82"
server_key_version: "encryptionkey@0"
I20260806 18:29:58.469877 13622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "c5aa7371d8234fd884a6daea95706f94"
format_stamp: "Formatted at 2026-08-06 18:29:58 on dist-test-slave-5bn2"
server_key: "af7964c20af7ff5edce2b0a3313905e4"
server_key_iv: "699ee2504ddfb961d7e4af9cb395ff82"
server_key_version: "encryptionkey@0"
I20260806 18:29:58.473672 13622 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:29:58.476224 13641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:58.477468 13622 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:29:58.477571 13622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "c5aa7371d8234fd884a6daea95706f94"
format_stamp: "Formatted at 2026-08-06 18:29:58 on dist-test-slave-5bn2"
server_key: "af7964c20af7ff5edce2b0a3313905e4"
server_key_iv: "699ee2504ddfb961d7e4af9cb395ff82"
server_key_version: "encryptionkey@0"
I20260806 18:29:58.477661 13622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:58.495496 13622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:58.498571 13622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:58.498734 13622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:58.499377 13622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:58.500303 13622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:58.500363 13622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:58.500412 13622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:58.500466 13622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:58.509389 13622 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:43225
I20260806 18:29:58.509414 13796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:43225 every 8 connection(s)
I20260806 18:29:58.510011 13622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:29:58.511400 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 13622
I20260806 18:29:58.511538 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:29:58.511855 7439 external_mini_cluster.cc:1428] Setting key 85534ee820ddd574f6c89a891b132fce
Aug 06 18:29:58 dist-test-slave-5bn2 krb5kdc[13463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040998, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.194@KRBTEST.COM; defaulting to no policy
I20260806 18:29:58.524360 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:58.511856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:49441 (local address 127.7.67.254:39289)
0806 18:29:58.512014 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:58.512018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:58.513487 (+ 1469us) server_negotiation.cc:407] Connection header received
0806 18:29:58.514409 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:58.514412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:58.514479 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:58.514551 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:58.515826 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:58.516361 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:58.517515 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:58.517758 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:58.517846 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:58.520810 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:58.520815 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:58.520818 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:58.520850 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:58.522329 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:58.522825 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:58.522829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:58.522830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:58.522952 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:58.523307 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:58.523310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:58.523311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:58.523489 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:58.523569 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:58.524111 (+ 542us) server_negotiation.cc:299] Negotiation successful
0806 18:29:58.524224 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.194@KRBTEST.COM" created.
I20260806 18:29:58.525381 13801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:58.512336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39289 (local address 127.7.67.193:49441)
0806 18:29:58.513325 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:58.513386 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:58.514202 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:58.514727 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:58.514735 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:58.515175 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:58.515678 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:58.515692 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:58.516485 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:58.516488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:58.517395 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:58.517406 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:58.517907 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:58.517911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:58.518089 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:58.518821 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:58.518869 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:58.520619 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:58.522441 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:58.522446 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:58.522458 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:58.522710 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:58.523072 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:58.523075 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:58.523077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:58.523233 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:58.523604 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:58.523609 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:58.523877 (+ 268us) client_negotiation.cc:769] Sending connection context
0806 18:29:58.524081 (+ 204us) client_negotiation.cc:240] Negotiation successful
0806 18:29:58.524373 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":127,"threads_started":1}
I20260806 18:29:58.526891 13798 heartbeater.cc:347] Connected to a master server at 127.7.67.254:39289
I20260806 18:29:58.527143 13798 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:58.527611 13798 heartbeater.cc:511] Master 127.7.67.254:39289 requested a full tablet report, sending...
I20260806 18:29:58.529307 13528 ts_manager.cc:194] Registered new tserver with Master: c5aa7371d8234fd884a6daea95706f94 (127.7.67.193:43225)
I20260806 18:29:58.530989 13528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:49441
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
I20260806 18:29:58.541698 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39289
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:36927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:58.654523 13809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:58.654834 13809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:58.654917 13809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:58.659049 13809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:58.659108 13809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:58.659175 13809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:29:58.664395 13809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:36927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:58.665459 13809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:58.666481 13809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:58.676091 13816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:58.676093 13821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:58.676091 13815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:58.676545 13809 server_base.cc:1034] running on GCE node
I20260806 18:29:58.676954 13809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:58.677593 13809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:58.678762 13809 hybrid_clock.cc:648] HybridClock initialized: now 1786040998678736 us; error 40 us; skew 500 ppm
Aug 06 18:29:58 dist-test-slave-5bn2 krb5kdc[13463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040998, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:58.681501 13809 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:29:58.682250 13809 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:58.682323 13809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:58.682564 13809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:58.684654 13809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "c4f3fec7b3f44a9c9ea230b2c9fdb5a2"
format_stamp: "Formatted at 2026-08-06 18:29:58 on dist-test-slave-5bn2"
server_key: "2dbab9620c866e8407d4df2552472dac"
server_key_iv: "d6bf73daaf775080f9d78ca9723fecca"
server_key_version: "encryptionkey@0"
I20260806 18:29:58.685178 13809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "c4f3fec7b3f44a9c9ea230b2c9fdb5a2"
format_stamp: "Formatted at 2026-08-06 18:29:58 on dist-test-slave-5bn2"
server_key: "2dbab9620c866e8407d4df2552472dac"
server_key_iv: "d6bf73daaf775080f9d78ca9723fecca"
server_key_version: "encryptionkey@0"
I20260806 18:29:58.688901 13809 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:29:58.691409 13828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:58.692652 13809 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:29:58.692773 13809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "c4f3fec7b3f44a9c9ea230b2c9fdb5a2"
format_stamp: "Formatted at 2026-08-06 18:29:58 on dist-test-slave-5bn2"
server_key: "2dbab9620c866e8407d4df2552472dac"
server_key_iv: "d6bf73daaf775080f9d78ca9723fecca"
server_key_version: "encryptionkey@0"
I20260806 18:29:58.692881 13809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:58.711884 13809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:58.714910 13809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:58.715111 13809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:58.715742 13809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:58.716635 13809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:58.716686 13809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:58.716741 13809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:58.716768 13809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:58.725593 13809 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:32971
I20260806 18:29:58.725613 13979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:32971 every 8 connection(s)
I20260806 18:29:58.726329 13809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:29:58.728942 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 13809
I20260806 18:29:58.729099 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:29:58.729415 7439 external_mini_cluster.cc:1428] Setting key 0790934826ac44ae2dfef50f786d0786
Aug 06 18:29:58 dist-test-slave-5bn2 krb5kdc[13463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040998, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.195@KRBTEST.COM; defaulting to no policy
I20260806 18:29:58.740535 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:58.728152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:59081 (local address 127.7.67.254:39289)
0806 18:29:58.728291 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:58.728295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:58.729626 (+ 1331us) server_negotiation.cc:407] Connection header received
0806 18:29:58.730614 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:58.730617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:58.730672 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:58.730748 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:58.731991 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:58.732535 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:58.733691 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:58.733906 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:58.734003 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:58.736902 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:58.736905 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:58.736907 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:58.736934 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:58.738249 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:58.738903 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:58.738906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:58.738907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:58.738984 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:58.739354 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:58.739356 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:58.739357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:58.739542 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:58.739612 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:58.740252 (+ 640us) server_negotiation.cc:299] Negotiation successful
0806 18:29:58.740385 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:29:58.741533 13984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:58.728546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39289 (local address 127.7.67.194:59081)
0806 18:29:58.729452 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:58.729513 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:58.730394 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:58.730966 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:58.730973 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:58.731339 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:58.731838 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:58.731852 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:58.732665 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:58.732668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:58.733563 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:58.733573 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:58.734060 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:58.734064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:58.734266 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:58.734904 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:58.734924 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:58.736707 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:58.738373 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:58.738380 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:58.738400 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:58.738781 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:58.739071 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:58.739074 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:58.739076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:58.739270 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:58.739642 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:58.739650 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:58.740011 (+ 361us) client_negotiation.cc:769] Sending connection context
0806 18:29:58.740224 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 18:29:58.740614 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":157,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.195@KRBTEST.COM" created.
I20260806 18:29:58.742890 13981 heartbeater.cc:347] Connected to a master server at 127.7.67.254:39289
I20260806 18:29:58.743166 13981 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:58.743757 13981 heartbeater.cc:511] Master 127.7.67.254:39289 requested a full tablet report, sending...
I20260806 18:29:58.745042 13528 ts_manager.cc:194] Registered new tserver with Master: c4f3fec7b3f44a9c9ea230b2c9fdb5a2 (127.7.67.194:32971)
I20260806 18:29:58.745810 13528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:59081
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
I20260806 18:29:58.761068 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39289
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:36927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:29:58.875614 13988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:58.875883 13988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:58.875928 13988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:58.879948 13988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:58.880007 13988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:58.880067 13988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:29:58.885413 13988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:36927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:58.886441 13988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:58.887452 13988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:58.897109 13999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:58.897137 13998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:58.897138 14001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:58.897300 13988 server_base.cc:1034] running on GCE node
I20260806 18:29:58.897831 13988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:58.898490 13988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:58.899653 13988 hybrid_clock.cc:648] HybridClock initialized: now 1786040998899626 us; error 52 us; skew 500 ppm
Aug 06 18:29:58 dist-test-slave-5bn2 krb5kdc[13463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040998, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:29:58.902344 13988 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:29:58.903155 13988 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:58.903230 13988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:58.903445 13988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:58.905550 13988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "eba8b93109c64851911e8dc229ecdf99"
format_stamp: "Formatted at 2026-08-06 18:29:58 on dist-test-slave-5bn2"
server_key: "0403b4c928d4ca2745709574621daeb6"
server_key_iv: "ffb40f6195d87940276b8e2d681f3249"
server_key_version: "encryptionkey@0"
I20260806 18:29:58.906117 13988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "eba8b93109c64851911e8dc229ecdf99"
format_stamp: "Formatted at 2026-08-06 18:29:58 on dist-test-slave-5bn2"
server_key: "0403b4c928d4ca2745709574621daeb6"
server_key_iv: "ffb40f6195d87940276b8e2d681f3249"
server_key_version: "encryptionkey@0"
I20260806 18:29:58.909945 13988 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:29:58.912456 14014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:58.913662 13988 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:29:58.913775 13988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "eba8b93109c64851911e8dc229ecdf99"
format_stamp: "Formatted at 2026-08-06 18:29:58 on dist-test-slave-5bn2"
server_key: "0403b4c928d4ca2745709574621daeb6"
server_key_iv: "ffb40f6195d87940276b8e2d681f3249"
server_key_version: "encryptionkey@0"
I20260806 18:29:58.913874 13988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:58.931386 13988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:58.934093 13988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:58.934268 13988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:58.934890 13988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:58.935740 13988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:58.935791 13988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:58.935843 13988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:58.935873 13988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:58.944355 13988 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:35225
I20260806 18:29:58.944376 14167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:35225 every 8 connection(s)
I20260806 18:29:58.944949 13988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:29:58.948122 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 13988
I20260806 18:29:58.948253 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:29:58.948608 7439 external_mini_cluster.cc:1428] Setting key 2e299ee302fee00d6f5abf5e4837849c
Aug 06 18:29:58 dist-test-slave-5bn2 krb5kdc[13463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786040998, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:29:58.958653 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:58.946794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:57851 (local address 127.7.67.254:39289)
0806 18:29:58.946951 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:58.946955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:58.948236 (+ 1281us) server_negotiation.cc:407] Connection header received
0806 18:29:58.949318 (+ 1082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:58.949321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:58.949378 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:58.949454 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:58.950924 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:58.951451 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:58.952537 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:58.952735 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:58.952800 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:58.955242 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:58.955245 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:58.955247 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:58.955273 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:58.956585 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:58.957103 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:58.957107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:58.957108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:58.957193 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:58.957526 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:58.957529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:58.957531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:58.957721 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:58.957806 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:58.958337 (+ 531us) server_negotiation.cc:299] Negotiation successful
0806 18:29:58.958476 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 18:29:58.959498 14172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:58.947184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39289 (local address 127.7.67.195:57851)
0806 18:29:58.948054 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:58.948115 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:58.949088 (+ 973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:58.949663 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:58.949672 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:58.950176 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:58.950754 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:58.950777 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:58.951548 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:58.951551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:58.952445 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:58.952456 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:58.952840 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:58.952842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:58.952991 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:58.953510 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:29:58.953526 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:58.955091 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:58.956704 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:58.956711 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:58.956727 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:58.956997 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:58.957275 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:29:58.957278 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:29:58.957281 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:29:58.957425 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:29:58.957822 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:58.957830 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:58.958094 (+ 264us) client_negotiation.cc:769] Sending connection context
0806 18:29:58.958259 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 18:29:58.958600 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":121,"threads_started":1}
I20260806 18:29:58.960628 14170 heartbeater.cc:347] Connected to a master server at 127.7.67.254:39289
I20260806 18:29:58.960858 14170 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:58.961395 14170 heartbeater.cc:511] Master 127.7.67.254:39289 requested a full tablet report, sending...
I20260806 18:29:58.962417 13527 ts_manager.cc:194] Registered new tserver with Master: eba8b93109c64851911e8dc229ecdf99 (127.7.67.195:35225)
I20260806 18:29:58.963033 13527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:57851
I20260806 18:29:58.974268 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:29:58.983229 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:58.976186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41394 (local address 127.7.67.254:39289)
0806 18:29:58.976297 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:58.976303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:58.976508 (+ 205us) server_negotiation.cc:407] Connection header received
0806 18:29:58.976639 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:58.976642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:58.976684 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:58.976748 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:58.977461 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:58.977976 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:58.978797 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:58.979065 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:58.979154 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:58.980415 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:58.980418 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:58.980420 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:29:58.980444 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:58.981544 (+ 1100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:58.981950 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:58.981953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:58.981955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:58.982044 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:29:58.982311 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:29:58.982315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:29:58.982316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:29:58.982558 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:58.982685 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:58.982942 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 18:29:58.983067 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 18:29:58.986052 13527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:29:58.989073 13527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:29:59.003597 14190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:58.998355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44620 (local address 127.7.67.193:43225)
0806 18:29:58.998686 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:58.998691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:58.998708 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:29:58.998768 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:58.998772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:58.998994 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:58.999189 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:58.999437 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.000351 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.001907 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.002759 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.002900 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:59.003084 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:59.003311 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 18:29:59.003415 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":132,"threads_started":1}
I20260806 18:29:59.003587 14186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:58.997647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:43225 (local address 127.0.0.1:44620)
0806 18:29:58.998385 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:58.998398 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:58.998480 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:58.999171 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:58.999174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:58.999195 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:58.999306 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:58.999313 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.000471 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.000474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:59.001763 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:59.001775 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.002913 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.002917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:59.003043 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:59.003152 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 18:29:59.003265 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 18:29:59.003413 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"mutex_wait_us":44,"thread_start_us":108,"threads_started":1}
I20260806 18:29:59.003929 14187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:58.997650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42816 (local address 127.7.67.194:32971)
0806 18:29:58.998483 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:58.998512 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:58.998538 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:29:58.998619 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:58.998624 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:58.998838 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:58.999433 (+ 595us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:58.999600 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.000711 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.002136 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.002932 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.003413 (+ 481us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:59.003559 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:59.003608 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 18:29:59.003781 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":147,"threads_started":1}
I20260806 18:29:59.004129 14185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:58.997526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:32971 (local address 127.0.0.1:42816)
0806 18:29:58.997979 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:58.998026 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:58.998150 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:58.999035 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:58.999041 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:58.999063 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:58.999226 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:58.999235 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.000849 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.000852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:59.002005 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:59.002015 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.003065 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.003069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:59.003193 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:59.003217 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:29:59.003265 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 18:29:59.003994 (+ 729us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":633,"thread_start_us":108,"threads_started":1}
I20260806 18:29:59.004312 14189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:58.998255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:35225 (local address 127.0.0.1:59014)
0806 18:29:58.999440 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:58.999453 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:58.999544 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:58.999963 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:58.999966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:58.999985 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:59.000068 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:59.000073 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.001209 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.001213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:59.002220 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:59.002230 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.003151 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.003154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:59.003238 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:59.003256 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:29:59.003317 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 18:29:59.004176 (+ 859us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1126,"mutex_wait_us":780,"thread_start_us":102,"threads_started":1}
I20260806 18:29:59.004519 14191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:58.998686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59014 (local address 127.7.67.195:35225)
0806 18:29:58.999056 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:58.999071 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:58.999548 (+ 477us) server_negotiation.cc:407] Connection header received
0806 18:29:58.999631 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:58.999636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:58.999813 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:58.999958 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:59.000160 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.001091 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.002334 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.002939 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.003050 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:59.003189 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:59.003395 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 18:29:59.003502 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":148,"threads_started":1}
I20260806 18:29:59.006892 13888 tablet_service.cc:1467] Processing CreateTablet for tablet ee552212f9944c66ba561e50fa7b9728 (DEFAULT_TABLE table=test-table [id=36ced4f8461546b1a87670607cebe956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:59.006976 13708 tablet_service.cc:1467] Processing CreateTablet for tablet ee552212f9944c66ba561e50fa7b9728 (DEFAULT_TABLE table=test-table [id=36ced4f8461546b1a87670607cebe956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:59.007176 14082 tablet_service.cc:1467] Processing CreateTablet for tablet ee552212f9944c66ba561e50fa7b9728 (DEFAULT_TABLE table=test-table [id=36ced4f8461546b1a87670607cebe956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:29:59.007985 13888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee552212f9944c66ba561e50fa7b9728. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:59.008087 14082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee552212f9944c66ba561e50fa7b9728. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:59.008080 13708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee552212f9944c66ba561e50fa7b9728. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:59.014803 14192 tablet_bootstrap.cc:492] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2: Bootstrap starting.
I20260806 18:29:59.016093 14193 tablet_bootstrap.cc:492] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99: Bootstrap starting.
I20260806 18:29:59.017241 14192 tablet_bootstrap.cc:654] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:59.017993 14193 tablet_bootstrap.cc:654] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:59.018349 14192 log.cc:824] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:29:59.019045 14193 log.cc:824] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99: Log is configured to *not* fsync() on all Append() calls
I20260806 18:29:59.019114 14194 tablet_bootstrap.cc:492] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94: Bootstrap starting.
I20260806 18:29:59.020782 14192 tablet_bootstrap.cc:492] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2: No bootstrap required, opened a new log
I20260806 18:29:59.020826 14193 tablet_bootstrap.cc:492] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99: No bootstrap required, opened a new log
I20260806 18:29:59.020991 14192 ts_tablet_manager.cc:1397] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:29:59.021003 14193 ts_tablet_manager.cc:1397] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:29:59.021847 14194 tablet_bootstrap.cc:654] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:59.023032 14194 log.cc:824] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94: Log is configured to *not* fsync() on all Append() calls
I20260806 18:29:59.024274 14193 raft_consensus.cc:348] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f3fec7b3f44a9c9ea230b2c9fdb5a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32971 } } peers { permanent_uuid: "c5aa7371d8234fd884a6daea95706f94" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43225 } } peers { permanent_uuid: "eba8b93109c64851911e8dc229ecdf99" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35225 } }
I20260806 18:29:59.024530 14193 raft_consensus.cc:374] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:59.024585 14193 raft_consensus.cc:729] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba8b93109c64851911e8dc229ecdf99, State: Initialized, Role: FOLLOWER
I20260806 18:29:59.024729 14192 raft_consensus.cc:348] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f3fec7b3f44a9c9ea230b2c9fdb5a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32971 } } peers { permanent_uuid: "c5aa7371d8234fd884a6daea95706f94" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43225 } } peers { permanent_uuid: "eba8b93109c64851911e8dc229ecdf99" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35225 } }
I20260806 18:29:59.025008 14192 raft_consensus.cc:374] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:59.025068 14192 raft_consensus.cc:729] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4f3fec7b3f44a9c9ea230b2c9fdb5a2, State: Initialized, Role: FOLLOWER
I20260806 18:29:59.025022 14193 consensus_queue.cc:260] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f3fec7b3f44a9c9ea230b2c9fdb5a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32971 } } peers { permanent_uuid: "c5aa7371d8234fd884a6daea95706f94" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43225 } } peers { permanent_uuid: "eba8b93109c64851911e8dc229ecdf99" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35225 } }
I20260806 18:29:59.025238 14194 tablet_bootstrap.cc:492] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94: No bootstrap required, opened a new log
I20260806 18:29:59.025439 14194 ts_tablet_manager.cc:1397] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 18:29:59.025563 14192 consensus_queue.cc:260] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f3fec7b3f44a9c9ea230b2c9fdb5a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32971 } } peers { permanent_uuid: "c5aa7371d8234fd884a6daea95706f94" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43225 } } peers { permanent_uuid: "eba8b93109c64851911e8dc229ecdf99" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35225 } }
I20260806 18:29:59.025763 14170 heartbeater.cc:503] Master 127.7.67.254:39289 was elected leader, sending a full tablet report...
I20260806 18:29:59.026086 14193 ts_tablet_manager.cc:1428] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:29:59.026551 14192 ts_tablet_manager.cc:1428] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260806 18:29:59.026898 13981 heartbeater.cc:503] Master 127.7.67.254:39289 was elected leader, sending a full tablet report...
I20260806 18:29:59.030992 14194 raft_consensus.cc:348] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f3fec7b3f44a9c9ea230b2c9fdb5a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32971 } } peers { permanent_uuid: "c5aa7371d8234fd884a6daea95706f94" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43225 } } peers { permanent_uuid: "eba8b93109c64851911e8dc229ecdf99" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35225 } }
I20260806 18:29:59.031229 14194 raft_consensus.cc:374] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:59.031275 14194 raft_consensus.cc:729] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5aa7371d8234fd884a6daea95706f94, State: Initialized, Role: FOLLOWER
I20260806 18:29:59.031638 14194 consensus_queue.cc:260] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f3fec7b3f44a9c9ea230b2c9fdb5a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32971 } } peers { permanent_uuid: "c5aa7371d8234fd884a6daea95706f94" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43225 } } peers { permanent_uuid: "eba8b93109c64851911e8dc229ecdf99" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35225 } }
I20260806 18:29:59.032183 13798 heartbeater.cc:503] Master 127.7.67.254:39289 was elected leader, sending a full tablet report...
I20260806 18:29:59.032480 14194 ts_tablet_manager.cc:1428] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 18:29:59.160084 14204 raft_consensus.cc:482] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:29:59.160342 14204 raft_consensus.cc:504] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f3fec7b3f44a9c9ea230b2c9fdb5a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32971 } } peers { permanent_uuid: "c5aa7371d8234fd884a6daea95706f94" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43225 } } peers { permanent_uuid: "eba8b93109c64851911e8dc229ecdf99" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35225 } }
I20260806 18:29:59.161597 14204 leader_election.cc:288] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4f3fec7b3f44a9c9ea230b2c9fdb5a2 (127.7.67.194:32971), eba8b93109c64851911e8dc229ecdf99 (127.7.67.195:35225)
I20260806 18:29:59.165832 14191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:59.162442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:49551 (local address 127.7.67.195:35225)
0806 18:29:59.162598 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:59.162602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:59.162759 (+ 157us) server_negotiation.cc:407] Connection header received
0806 18:29:59.162880 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:59.162883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:59.162939 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:59.163001 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:59.163257 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.163806 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.164834 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.165231 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.165337 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:59.165405 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:59.165648 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 18:29:59.165720 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:29:59.165833 14187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:59.162006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:44689 (local address 127.7.67.194:32971)
0806 18:29:59.162140 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:59.162144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:59.162289 (+ 145us) server_negotiation.cc:407] Connection header received
0806 18:29:59.162461 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:59.162465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:59.162525 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:59.162617 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:29:59.162955 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.163577 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.164793 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.165224 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.165333 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:59.165391 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:59.165642 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 18:29:59.165714 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:29:59.165848 14207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:59.162304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:35225 (local address 127.7.67.193:49551)
0806 18:29:59.162669 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:59.162683 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:59.162779 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:59.163028 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:59.163030 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:59.163047 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:59.163160 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:59.163166 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.163912 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.163915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:59.164706 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:59.164716 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.165356 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.165360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:59.165504 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:59.165547 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 18:29:59.165614 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 18:29:59.165688 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":34,"thread_start_us":112,"threads_started":1}
I20260806 18:29:59.165853 14206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:59.161863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:32971 (local address 127.7.67.193:44689)
0806 18:29:59.162197 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:59.162211 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:59.162335 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:59.162645 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:59.162648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:59.162669 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:29:59.162822 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:59.162830 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.163692 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.163695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:59.164658 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:29:59.164670 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.165360 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.165364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:29:59.165511 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:59.165542 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 18:29:59.165605 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 18:29:59.165680 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":83,"thread_start_us":120,"threads_started":1}
I20260806 18:29:59.166765 14109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee552212f9944c66ba561e50fa7b9728" candidate_uuid: "c5aa7371d8234fd884a6daea95706f94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba8b93109c64851911e8dc229ecdf99" is_pre_election: true
I20260806 18:29:59.166765 13915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee552212f9944c66ba561e50fa7b9728" candidate_uuid: "c5aa7371d8234fd884a6daea95706f94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4f3fec7b3f44a9c9ea230b2c9fdb5a2" is_pre_election: true
I20260806 18:29:59.167150 14109 raft_consensus.cc:2454] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5aa7371d8234fd884a6daea95706f94 in term 0.
I20260806 18:29:59.167150 13915 raft_consensus.cc:2454] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5aa7371d8234fd884a6daea95706f94 in term 0.
I20260806 18:29:59.167636 13642 leader_election.cc:302] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4f3fec7b3f44a9c9ea230b2c9fdb5a2, c5aa7371d8234fd884a6daea95706f94; no voters:
I20260806 18:29:59.167884 14204 raft_consensus.cc:2781] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:29:59.167976 14204 raft_consensus.cc:482] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:29:59.168020 14204 raft_consensus.cc:3037] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:59.169087 14204 raft_consensus.cc:504] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f3fec7b3f44a9c9ea230b2c9fdb5a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32971 } } peers { permanent_uuid: "c5aa7371d8234fd884a6daea95706f94" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43225 } } peers { permanent_uuid: "eba8b93109c64851911e8dc229ecdf99" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35225 } }
I20260806 18:29:59.169413 14204 leader_election.cc:288] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [CANDIDATE]: Term 1 election: Requested vote from peers c4f3fec7b3f44a9c9ea230b2c9fdb5a2 (127.7.67.194:32971), eba8b93109c64851911e8dc229ecdf99 (127.7.67.195:35225)
I20260806 18:29:59.169804 13915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee552212f9944c66ba561e50fa7b9728" candidate_uuid: "c5aa7371d8234fd884a6daea95706f94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4f3fec7b3f44a9c9ea230b2c9fdb5a2"
I20260806 18:29:59.169835 14109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee552212f9944c66ba561e50fa7b9728" candidate_uuid: "c5aa7371d8234fd884a6daea95706f94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba8b93109c64851911e8dc229ecdf99"
I20260806 18:29:59.169955 13915 raft_consensus.cc:3037] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:59.169950 14109 raft_consensus.cc:3037] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:59.171407 13915 raft_consensus.cc:2454] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5aa7371d8234fd884a6daea95706f94 in term 1.
I20260806 18:29:59.171411 14109 raft_consensus.cc:2454] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5aa7371d8234fd884a6daea95706f94 in term 1.
I20260806 18:29:59.171711 13642 leader_election.cc:302] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4f3fec7b3f44a9c9ea230b2c9fdb5a2, c5aa7371d8234fd884a6daea95706f94; no voters:
I20260806 18:29:59.171873 14204 raft_consensus.cc:2781] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:59.172094 14204 raft_consensus.cc:686] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [term 1 LEADER]: Becoming Leader. State: Replica: c5aa7371d8234fd884a6daea95706f94, State: Running, Role: LEADER
I20260806 18:29:59.172420 14204 consensus_queue.cc:237] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f3fec7b3f44a9c9ea230b2c9fdb5a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32971 } } peers { permanent_uuid: "c5aa7371d8234fd884a6daea95706f94" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43225 } } peers { permanent_uuid: "eba8b93109c64851911e8dc229ecdf99" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35225 } }
I20260806 18:29:59.175897 13528 catalog_manager.cc:5495] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5aa7371d8234fd884a6daea95706f94 (127.7.67.193). New cstate: current_term: 1 leader_uuid: "c5aa7371d8234fd884a6daea95706f94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f3fec7b3f44a9c9ea230b2c9fdb5a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5aa7371d8234fd884a6daea95706f94" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eba8b93109c64851911e8dc229ecdf99" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35225 } health_report { overall_health: UNKNOWN } } }
I20260806 18:29:59.192782 14190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:59.189234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44624 (local address 127.7.67.193:43225)
0806 18:29:59.189371 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:59.189375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:59.189390 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:29:59.189502 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:59.189505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:59.189564 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:59.189657 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:29:59.189897 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.190418 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.191367 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:29:59.191583 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:29:59.191672 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:29:59.191958 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:29:59.192390 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:29:59.192499 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:59.192581 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 18:29:59.192640 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 18:29:59.195863 14171 tablet.cc:2369] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:29:59.199003 13915 raft_consensus.cc:1261] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2 [term 1 FOLLOWER]: Refusing update from remote peer c5aa7371d8234fd884a6daea95706f94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:29:59.199002 14109 raft_consensus.cc:1261] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99 [term 1 FOLLOWER]: Refusing update from remote peer c5aa7371d8234fd884a6daea95706f94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:29:59.199682 14204 consensus_queue.cc:1035] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4f3fec7b3f44a9c9ea230b2c9fdb5a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:29:59.199890 14208 consensus_queue.cc:1035] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eba8b93109c64851911e8dc229ecdf99" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:29:59.206262 14211 mvcc.cc:205] Tried to move back new op lower bound from 7315623932711436288 to 7315623932613029888. Current Snapshot: MvccSnapshot[applied={T|T < 7315623932711436288}]
I20260806 18:29:59.206765 14212 mvcc.cc:205] Tried to move back new op lower bound from 7315623932711436288 to 7315623932613029888. Current Snapshot: MvccSnapshot[applied={T|T < 7315623932711436288}]
I20260806 18:29:59.216316 13528 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:29:59.216514 13528 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:29:59.219012 13528 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74796f2c1a3f4946b0edc51a10b1a5f5: Sending DeleteTablet for 3 replicas of tablet ee552212f9944c66ba561e50fa7b9728
I20260806 18:29:59.219827 13708 tablet_service.cc:1514] Processing DeleteTablet for tablet ee552212f9944c66ba561e50fa7b9728 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:59 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:44620
I20260806 18:29:59.219817 14082 tablet_service.cc:1514] Processing DeleteTablet for tablet ee552212f9944c66ba561e50fa7b9728 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:59 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:59014
I20260806 18:29:59.219888 13888 tablet_service.cc:1514] Processing DeleteTablet for tablet ee552212f9944c66ba561e50fa7b9728 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:29:59 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:42816
I20260806 18:29:59.220315 14228 tablet_replica.cc:330] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94: stopping tablet replica
I20260806 18:29:59.220783 14228 raft_consensus.cc:2229] T ee552212f9944c66ba561e50fa7b9728 P c5aa7371d8234fd884a6daea95706f94 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:29:59.220944 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 13622
I20260806 18:29:59.231225 14229 tablet_replica.cc:330] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99: stopping tablet replica
I20260806 18:29:59.231635 14229 raft_consensus.cc:2229] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:29:59.232014 14229 raft_consensus.cc:2258] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:29:59.232453 14230 tablet_replica.cc:330] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2: stopping tablet replica
I20260806 18:29:59.232759 14230 raft_consensus.cc:2229] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:29:59.233057 14230 raft_consensus.cc:2258] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:29:59.234463 14229 ts_tablet_manager.cc:1905] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:29:59.234936 14230 ts_tablet_manager.cc:1905] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:29:59.237488 14229 ts_tablet_manager.cc:1918] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:29:59.237673 14229 log.cc:1196] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/ee552212f9944c66ba561e50fa7b9728
I20260806 18:29:59.237725 14230 ts_tablet_manager.cc:1918] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:29:59.237869 14230 log.cc:1196] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/ee552212f9944c66ba561e50fa7b9728
I20260806 18:29:59.238139 14229 ts_tablet_manager.cc:1939] T ee552212f9944c66ba561e50fa7b9728 P eba8b93109c64851911e8dc229ecdf99: Deleting consensus metadata
I20260806 18:29:59.238191 14230 ts_tablet_manager.cc:1939] T ee552212f9944c66ba561e50fa7b9728 P c4f3fec7b3f44a9c9ea230b2c9fdb5a2: Deleting consensus metadata
I20260806 18:29:59.239518 13506 catalog_manager.cc:4841] TS c4f3fec7b3f44a9c9ea230b2c9fdb5a2 (127.7.67.194:32971): tablet ee552212f9944c66ba561e50fa7b9728 (table test-table [id=36ced4f8461546b1a87670607cebe956]) successfully deleted
W20260806 18:29:59.239605 13513 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.193:43225 (error 108)
I20260806 18:29:59.239718 13506 catalog_manager.cc:4841] TS eba8b93109c64851911e8dc229ecdf99 (127.7.67.195:35225): tablet ee552212f9944c66ba561e50fa7b9728 (table test-table [id=36ced4f8461546b1a87670607cebe956]) successfully deleted
I20260806 18:29:59.240455 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 13809
I20260806 18:29:59.240666 14189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:59.240364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:43225 (local address 127.0.0.1:44628)
0806 18:29:59.240510 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:59.240582 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:43225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 18:29:59.251626 13513 catalog_manager.cc:4584] TS c5aa7371d8234fd884a6daea95706f94 (127.7.67.193:43225): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ee552212f9944c66ba561e50fa7b9728: Network error: Client connection negotiation failed: client connection to 127.7.67.193:43225: connect: Connection refused (error 111)
I20260806 18:29:59.261719 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 13988
I20260806 18:29:59.276242 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 13483
2026-08-06T18:29:59Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2478 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T18:29:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:29:59Z Disabled control of system clock
I20260806 18:29:59.313495 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:40373
--webserver_interface=127.7.67.254
--webserver_port=0
--builtin_ntp_servers=127.7.67.212:45499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:40373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 18:29:59.426504 14236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:59.426805 14236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:59.426867 14236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:59.430944 14236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:29:59.431010 14236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:59.431022 14236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:59.431041 14236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:29:59.431070 14236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:29:59.435931 14236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:45499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:40373
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:40373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:59.437108 14236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:59.438563 14236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:59.446944 14247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:59.446972 14243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:59.447055 14236 server_base.cc:1034] running on GCE node
W20260806 18:29:59.446939 14245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:59.447861 14236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:59.448988 14236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:59.450162 14236 hybrid_clock.cc:648] HybridClock initialized: now 1786040999450141 us; error 32 us; skew 500 ppm
I20260806 18:29:59.452013 14236 webserver.cc:460] Webserver started at http://127.7.67.254:44239/ using document root <none> and password file <none>
I20260806 18:29:59.452586 14236 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:59.452657 14236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:59.452865 14236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:59.455001 14236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "5239497d90ec43a8b3f67981559ef569"
format_stamp: "Formatted at 2026-08-06 18:29:59 on dist-test-slave-5bn2"
server_key: "9be715290ce08bb26ea68f93d12166d9"
server_key_iv: "655276dfba915be367d5573a1db54954"
server_key_version: "encryptionkey@0"
I20260806 18:29:59.455571 14236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "5239497d90ec43a8b3f67981559ef569"
format_stamp: "Formatted at 2026-08-06 18:29:59 on dist-test-slave-5bn2"
server_key: "9be715290ce08bb26ea68f93d12166d9"
server_key_iv: "655276dfba915be367d5573a1db54954"
server_key_version: "encryptionkey@0"
I20260806 18:29:59.459280 14236 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260806 18:29:59.461740 14260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:59.462968 14236 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:29:59.463096 14236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "5239497d90ec43a8b3f67981559ef569"
format_stamp: "Formatted at 2026-08-06 18:29:59 on dist-test-slave-5bn2"
server_key: "9be715290ce08bb26ea68f93d12166d9"
server_key_iv: "655276dfba915be367d5573a1db54954"
server_key_version: "encryptionkey@0"
I20260806 18:29:59.463199 14236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:59.467862 14236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:59.468489 14236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:59.468655 14236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:59.475392 14236 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:40373
I20260806 18:29:59.475397 14329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:40373 every 8 connection(s)
I20260806 18:29:59.476356 14236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:29:59.478480 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 14236
I20260806 18:29:59.478590 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:29:59.478910 7439 external_mini_cluster.cc:1428] Setting key b1cd3f0326caa198448ca5b9fb0b4cf3
I20260806 18:29:59.479928 14330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:29:59.485276 14333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:59.480389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48896 (local address 127.7.67.254:40373)
0806 18:29:59.481110 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:59.481131 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:59.481175 (+ 44us) server_negotiation.cc:407] Connection header received
0806 18:29:59.482042 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:59.482069 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:59.482530 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:59.482958 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:59.483735 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:59.483742 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:59.483757 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:29:59.483769 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:59.483840 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:29:59.483864 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:59.484021 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:59.484089 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 18:29:59.484337 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":131,"threads_started":1}
I20260806 18:29:59.487190 14330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569: Bootstrap starting.
I20260806 18:29:59.489517 14330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569: Neither blocks nor log segments found. Creating new log.
I20260806 18:29:59.490275 14330 log.cc:824] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569: Log is configured to *not* fsync() on all Append() calls
I20260806 18:29:59.492239 14330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569: No bootstrap required, opened a new log
I20260806 18:29:59.495858 14330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5239497d90ec43a8b3f67981559ef569" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40373 } }
I20260806 18:29:59.496105 14330 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:29:59.496165 14330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5239497d90ec43a8b3f67981559ef569, State: Initialized, Role: FOLLOWER
I20260806 18:29:59.496585 14330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5239497d90ec43a8b3f67981559ef569" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40373 } }
I20260806 18:29:59.496721 14330 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:29:59.496785 14330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:29:59.496887 14330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:29:59.497792 14330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5239497d90ec43a8b3f67981559ef569" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40373 } }
I20260806 18:29:59.498158 14330 leader_election.cc:302] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5239497d90ec43a8b3f67981559ef569; no voters:
I20260806 18:29:59.498535 14330 leader_election.cc:288] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:29:59.498643 14335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:29:59.498890 14335 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569 [term 1 LEADER]: Becoming Leader. State: Replica: 5239497d90ec43a8b3f67981559ef569, State: Running, Role: LEADER
I20260806 18:29:59.499178 14335 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5239497d90ec43a8b3f67981559ef569" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40373 } }
I20260806 18:29:59.499747 14330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:29:59.500816 14337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5239497d90ec43a8b3f67981559ef569" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5239497d90ec43a8b3f67981559ef569" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40373 } } }
I20260806 18:29:59.500816 14338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5239497d90ec43a8b3f67981559ef569. Latest consensus state: current_term: 1 leader_uuid: "5239497d90ec43a8b3f67981559ef569" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5239497d90ec43a8b3f67981559ef569" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40373 } } }
I20260806 18:29:59.501056 14338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569 [sys.catalog]: This master's current role is: LEADER
I20260806 18:29:59.501001 14337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569 [sys.catalog]: This master's current role is: LEADER
I20260806 18:29:59.501634 14349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:29:59.505067 14349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:29:59.510717 14349 catalog_manager.cc:1347] Generated new cluster ID: 34bfe5eb3ccb442f9355b327d1da4357
I20260806 18:29:59.510785 14349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:29:59.518788 14349 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:29:59.519140 14349 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:29:59.524171 14349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569: Generated new TSK 0
I20260806 18:29:59.524888 14349 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:29:59.535704 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:40373
--builtin_ntp_servers=127.7.67.212:45499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 18:29:59.650663 14367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:59.650946 14367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:59.650986 14367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:59.655051 14367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:59.655112 14367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:59.655166 14367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:29:59.659876 14367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:45499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:40373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:59.660935 14367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:59.662366 14367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:59.672609 14377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:59.672623 14375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:59.672626 14373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:59.672784 14367 server_base.cc:1034] running on GCE node
I20260806 18:29:59.673262 14367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:59.673869 14367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:59.675016 14367 hybrid_clock.cc:648] HybridClock initialized: now 1786040999674995 us; error 30 us; skew 500 ppm
I20260806 18:29:59.676784 14367 webserver.cc:460] Webserver started at http://127.7.67.193:37245/ using document root <none> and password file <none>
I20260806 18:29:59.677349 14367 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:59.677414 14367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:59.677627 14367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:59.679754 14367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "fb02aaffab8341c5a7106692e01bcc1d"
format_stamp: "Formatted at 2026-08-06 18:29:59 on dist-test-slave-5bn2"
server_key: "e7a32e13813f8a81a93d30885f37d6c7"
server_key_iv: "18c1969cfdd65f0f066d882bc0f93f2b"
server_key_version: "encryptionkey@0"
I20260806 18:29:59.680310 14367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "fb02aaffab8341c5a7106692e01bcc1d"
format_stamp: "Formatted at 2026-08-06 18:29:59 on dist-test-slave-5bn2"
server_key: "e7a32e13813f8a81a93d30885f37d6c7"
server_key_iv: "18c1969cfdd65f0f066d882bc0f93f2b"
server_key_version: "encryptionkey@0"
I20260806 18:29:59.684039 14367 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:29:59.686419 14384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:59.687654 14367 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:29:59.687767 14367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "fb02aaffab8341c5a7106692e01bcc1d"
format_stamp: "Formatted at 2026-08-06 18:29:59 on dist-test-slave-5bn2"
server_key: "e7a32e13813f8a81a93d30885f37d6c7"
server_key_iv: "18c1969cfdd65f0f066d882bc0f93f2b"
server_key_version: "encryptionkey@0"
I20260806 18:29:59.687865 14367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:59.692456 14367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:59.693055 14367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:59.693214 14367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:59.693763 14367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:59.694682 14367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:59.694730 14367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:59.694783 14367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:59.694808 14367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:59.703455 14367 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:43019
I20260806 18:29:59.703485 14562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:43019 every 8 connection(s)
I20260806 18:29:59.704358 14367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:29:59.710214 14333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:59.706189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:53051 (local address 127.7.67.254:40373)
0806 18:29:59.706327 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:59.706330 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:59.707441 (+ 1111us) server_negotiation.cc:407] Connection header received
0806 18:29:59.708769 (+ 1328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:59.708772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:59.708808 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:59.708880 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:59.709886 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:59.709889 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:59.709891 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:29:59.709898 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:59.709911 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:29:59.709917 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:59.709979 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:59.710057 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 18:29:59.710101 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:29:59.711071 14568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:59.706589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:40373 (local address 127.7.67.193:53051)
0806 18:29:59.707282 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:59.707323 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:59.708593 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:59.709031 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:59.709038 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:59.709125 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:59.709666 (+ 541us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:29:59.709668 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:59.709710 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:29:59.709779 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:59.709965 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:59.709970 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:59.709973 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:29:59.710102 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 18:29:59.710286 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":121,"threads_started":1}
I20260806 18:29:59.712270 14563 heartbeater.cc:347] Connected to a master server at 127.7.67.254:40373
I20260806 18:29:59.712486 14563 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:59.712908 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 14367
I20260806 18:29:59.712924 14563 heartbeater.cc:511] Master 127.7.67.254:40373 requested a full tablet report, sending...
I20260806 18:29:59.713043 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:29:59.713342 7439 external_mini_cluster.cc:1428] Setting key cd890439ab15a0ab83171aa2751dfced
I20260806 18:29:59.714553 14280 ts_manager.cc:194] Registered new tserver with Master: fb02aaffab8341c5a7106692e01bcc1d (127.7.67.193:43019)
I20260806 18:29:59.715338 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:40373
--builtin_ntp_servers=127.7.67.212:45499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 18:29:59.829273 14569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:29:59.829564 14569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:29:59.829620 14569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:29:59.833636 14569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:29:59.833695 14569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:29:59.833770 14569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:29:59.838513 14569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:45499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:40373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:29:59.839641 14569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:29:59.841055 14569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:29:59.850704 14579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:29:59.850735 14577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:59.850749 14569 server_base.cc:1034] running on GCE node
W20260806 18:29:59.850757 14575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:29:59.851374 14569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:29:59.852053 14569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:29:59.853231 14569 hybrid_clock.cc:648] HybridClock initialized: now 1786040999853200 us; error 41 us; skew 500 ppm
I20260806 18:29:59.855136 14569 webserver.cc:460] Webserver started at http://127.7.67.194:37161/ using document root <none> and password file <none>
I20260806 18:29:59.855719 14569 fs_manager.cc:359] Metadata directory not provided
I20260806 18:29:59.855791 14569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:29:59.855988 14569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:29:59.858080 14569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "261ed9e8e13c4d50b2277b60aa73c80c"
format_stamp: "Formatted at 2026-08-06 18:29:59 on dist-test-slave-5bn2"
server_key: "5f56c5480ed0d77a66159d7ab6d94d83"
server_key_iv: "7d7f7d3ed9231caeabf7481c0a43a8d3"
server_key_version: "encryptionkey@0"
I20260806 18:29:59.858634 14569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "261ed9e8e13c4d50b2277b60aa73c80c"
format_stamp: "Formatted at 2026-08-06 18:29:59 on dist-test-slave-5bn2"
server_key: "5f56c5480ed0d77a66159d7ab6d94d83"
server_key_iv: "7d7f7d3ed9231caeabf7481c0a43a8d3"
server_key_version: "encryptionkey@0"
I20260806 18:29:59.862366 14569 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:29:59.864817 14586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:59.866073 14569 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:29:59.866182 14569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "261ed9e8e13c4d50b2277b60aa73c80c"
format_stamp: "Formatted at 2026-08-06 18:29:59 on dist-test-slave-5bn2"
server_key: "5f56c5480ed0d77a66159d7ab6d94d83"
server_key_iv: "7d7f7d3ed9231caeabf7481c0a43a8d3"
server_key_version: "encryptionkey@0"
I20260806 18:29:59.866279 14569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:29:59.871129 14569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:29:59.871731 14569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:29:59.871889 14569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:29:59.872423 14569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:29:59.873293 14569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:29:59.873347 14569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:59.873385 14569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:29:59.873406 14569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:29:59.882282 14569 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:39493
I20260806 18:29:59.882318 14763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:39493 every 8 connection(s)
I20260806 18:29:59.883437 14569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:29:59.889637 14333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:59.885408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:50479 (local address 127.7.67.254:40373)
0806 18:29:59.885553 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:29:59.885557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:29:59.886654 (+ 1097us) server_negotiation.cc:407] Connection header received
0806 18:29:59.888059 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:29:59.888063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:29:59.888109 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:29:59.888180 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:29:59.889187 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:29:59.889190 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:29:59.889193 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:29:59.889205 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:29:59.889226 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:29:59.889234 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:29:59.889316 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 18:29:59.889450 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 18:29:59.889497 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:29:59.890465 14766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:29:59.885840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:40373 (local address 127.7.67.194:50479)
0806 18:29:59.886510 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0806 18:29:59.886551 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:29:59.887837 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:29:59.888339 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:29:59.888346 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:29:59.888435 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:29:59.888937 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:29:59.888938 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:29:59.888977 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:29:59.889044 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:29:59.889310 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:29:59.889318 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:29:59.889324 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:29:59.889480 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 18:29:59.889711 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":128,"threads_started":1}
I20260806 18:29:59.891595 14764 heartbeater.cc:347] Connected to a master server at 127.7.67.254:40373
I20260806 18:29:59.891870 14764 heartbeater.cc:464] Registering TS with master...
I20260806 18:29:59.892273 14764 heartbeater.cc:511] Master 127.7.67.254:40373 requested a full tablet report, sending...
I20260806 18:29:59.892349 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 14569
I20260806 18:29:59.892480 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:29:59.892745 7439 external_mini_cluster.cc:1428] Setting key 757cef6224fafd504c3fb7509cf367a9
I20260806 18:29:59.893428 14280 ts_manager.cc:194] Registered new tserver with Master: 261ed9e8e13c4d50b2277b60aa73c80c (127.7.67.194:39493)
I20260806 18:29:59.894704 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:40373
--builtin_ntp_servers=127.7.67.212:45499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 18:30:00.007952 14767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:00.008224 14767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:00.008261 14767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:00.012253 14767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:00.012310 14767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:00.012365 14767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:30:00.017184 14767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:45499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:40373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:00.018244 14767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:00.019665 14767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:00.029322 14778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:00.029548 14767 server_base.cc:1034] running on GCE node
W20260806 18:30:00.029322 14776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:00.029327 14775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:00.030097 14767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:00.030745 14767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:00.031904 14767 hybrid_clock.cc:648] HybridClock initialized: now 1786041000031879 us; error 44 us; skew 500 ppm
I20260806 18:30:00.033722 14767 webserver.cc:460] Webserver started at http://127.7.67.195:38421/ using document root <none> and password file <none>
I20260806 18:30:00.034299 14767 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:00.034369 14767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:00.034569 14767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:00.036697 14767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "506b453190034de99008c19eda99a34b"
format_stamp: "Formatted at 2026-08-06 18:30:00 on dist-test-slave-5bn2"
server_key: "a6feebf229fdecf622bef22255a99b17"
server_key_iv: "fb9a78d20b8da6dc5749ac698df3d3a8"
server_key_version: "encryptionkey@0"
I20260806 18:30:00.037279 14767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "506b453190034de99008c19eda99a34b"
format_stamp: "Formatted at 2026-08-06 18:30:00 on dist-test-slave-5bn2"
server_key: "a6feebf229fdecf622bef22255a99b17"
server_key_iv: "fb9a78d20b8da6dc5749ac698df3d3a8"
server_key_version: "encryptionkey@0"
I20260806 18:30:00.041190 14767 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:30:00.043648 14786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:00.044889 14767 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:30:00.045006 14767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "506b453190034de99008c19eda99a34b"
format_stamp: "Formatted at 2026-08-06 18:30:00 on dist-test-slave-5bn2"
server_key: "a6feebf229fdecf622bef22255a99b17"
server_key_iv: "fb9a78d20b8da6dc5749ac698df3d3a8"
server_key_version: "encryptionkey@0"
I20260806 18:30:00.045104 14767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:00.049732 14767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:00.050351 14767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:00.050498 14767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:00.051061 14767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:00.052021 14767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:00.052068 14767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:00.052107 14767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:00.052127 14767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:00.060693 14767 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:36557
I20260806 18:30:00.060739 14955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:36557 every 8 connection(s)
I20260806 18:30:00.061650 14767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:30:00.067713 14333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:00.063492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:50439 (local address 127.7.67.254:40373)
0806 18:30:00.063625 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:00.063628 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:00.064684 (+ 1056us) server_negotiation.cc:407] Connection header received
0806 18:30:00.066007 (+ 1323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:00.066011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:00.066059 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:00.066134 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:00.067260 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:00.067264 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:00.067267 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:00.067280 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:00.067302 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:00.067310 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:00.067407 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:00.067522 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 18:30:00.067568 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:30:00.068549 14960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:00.063847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:40373 (local address 127.7.67.195:50439)
0806 18:30:00.064536 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:00.064578 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:00.065784 (+ 1206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:00.066313 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:00.066322 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:00.066413 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:00.066999 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:30:00.067001 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:00.067042 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:30:00.067111 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:00.067380 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:00.067388 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:00.067395 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:30:00.067565 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 18:30:00.067807 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":107,"threads_started":1}
I20260806 18:30:00.069625 14956 heartbeater.cc:347] Connected to a master server at 127.7.67.254:40373
I20260806 18:30:00.069836 14956 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:00.070246 14956 heartbeater.cc:511] Master 127.7.67.254:40373 requested a full tablet report, sending...
I20260806 18:30:00.071300 14279 ts_manager.cc:194] Registered new tserver with Master: 506b453190034de99008c19eda99a34b (127.7.67.195:36557)
I20260806 18:30:00.071911 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 14767
I20260806 18:30:00.072041 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:30:00.072345 7439 external_mini_cluster.cc:1428] Setting key 8cd4c1d803d7c6dc0894d8087f83b13d
I20260806 18:30:00.074172 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:30:00.077505 14333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:00.076015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48898 (local address 127.7.67.254:40373)
0806 18:30:00.076147 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:00.076152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:00.076304 (+ 152us) server_negotiation.cc:407] Connection header received
0806 18:30:00.076408 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:00.076410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:00.076443 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:00.076506 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:00.077165 (+ 659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:00.077168 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:00.077170 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:00.077180 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:00.077198 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:00.077206 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:00.077285 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:00.077348 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 18:30:00.077380 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:30:00.080165 14279 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:30:00.083258 14279 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:30:00.094449 14971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:00.091556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:36557 (local address 127.0.0.1:35590)
0806 18:30:00.092297 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:00.092355 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:00.092486 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:00.093003 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:00.093009 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:00.093088 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:00.093643 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:30:00.093645 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:00.093665 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:30:00.093737 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:00.094089 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:00.094099 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:00.094106 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:30:00.094223 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 18:30:00.094299 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":104,"threads_started":1}
I20260806 18:30:00.094522 14973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:00.091702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35590 (local address 127.7.67.195:36557)
0806 18:30:00.092557 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:00.092570 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:00.092599 (+ 29us) server_negotiation.cc:407] Connection header received
0806 18:30:00.092656 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:00.092662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:00.092847 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:00.092980 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:00.093880 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:00.093887 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:00.093900 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:00.093911 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:00.093957 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:00.093993 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:00.094100 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:00.094258 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 18:30:00.094334 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":164,"threads_started":1}
I20260806 18:30:00.095103 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:00.091693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:39493 (local address 127.0.0.1:52260)
0806 18:30:00.093064 (+ 1371us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:00.093076 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:00.093156 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:00.093906 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:00.093909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:00.093924 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:00.094337 (+ 413us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:30:00.094338 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:00.094348 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:30:00.094373 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:00.094885 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:00.094888 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:00.094892 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:30:00.094974 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 18:30:00.095009 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1310,"thread_start_us":110,"threads_started":1}
I20260806 18:30:00.096448 14972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:00.091934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:43019 (local address 127.0.0.1:56108)
0806 18:30:00.094506 (+ 2572us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:00.094520 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:00.094896 (+ 376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:00.095306 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:00.095309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:00.095323 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:00.095693 (+ 370us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:30:00.095695 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:00.095705 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:30:00.095731 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:00.096224 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:00.096227 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:00.096230 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:30:00.096298 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 18:30:00.096334 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2511,"thread_start_us":86,"threads_started":1}
I20260806 18:30:00.096791 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:00.092460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56108 (local address 127.7.67.193:43019)
0806 18:30:00.093388 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:00.093397 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:00.094810 (+ 1413us) server_negotiation.cc:407] Connection header received
0806 18:30:00.095005 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:00.095011 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:00.095191 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:00.095316 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:00.096058 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:00.096064 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:00.096075 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:00.096086 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:00.096121 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:00.096140 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:00.096234 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:00.096540 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 18:30:00.096623 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":824,"thread_start_us":160,"threads_started":1}
I20260806 18:30:00.097783 14860 tablet_service.cc:1467] Processing CreateTablet for tablet 19f985963f3a4cceb6824b8e5b828a17 (DEFAULT_TABLE table=test-table [id=9d62752b66644a8fb3186c80a9ec316b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:00.099043 14860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f985963f3a4cceb6824b8e5b828a17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:00.099267 14975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:00.092443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52260 (local address 127.7.67.194:39493)
0806 18:30:00.092800 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:00.092812 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:00.093435 (+ 623us) server_negotiation.cc:407] Connection header received
0806 18:30:00.093487 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:00.093492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:00.093676 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:00.093815 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:00.094659 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:00.094665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:00.094677 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:00.094690 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:00.094733 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:00.094772 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:00.094903 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:00.099024 (+ 4121us) server_negotiation.cc:299] Negotiation successful
0806 18:30:00.099102 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":151,"threads_started":1}
I20260806 18:30:00.099531 14466 tablet_service.cc:1467] Processing CreateTablet for tablet 19f985963f3a4cceb6824b8e5b828a17 (DEFAULT_TABLE table=test-table [id=9d62752b66644a8fb3186c80a9ec316b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:00.100595 14466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f985963f3a4cceb6824b8e5b828a17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:00.101733 14658 tablet_service.cc:1467] Processing CreateTablet for tablet 19f985963f3a4cceb6824b8e5b828a17 (DEFAULT_TABLE table=test-table [id=9d62752b66644a8fb3186c80a9ec316b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:00.102591 14658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f985963f3a4cceb6824b8e5b828a17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:00.106945 14977 tablet_bootstrap.cc:492] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b: Bootstrap starting.
I20260806 18:30:00.108335 14978 tablet_bootstrap.cc:492] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c: Bootstrap starting.
I20260806 18:30:00.109254 14981 tablet_bootstrap.cc:492] T 19f985963f3a4cceb6824b8e5b828a17 P fb02aaffab8341c5a7106692e01bcc1d: Bootstrap starting.
I20260806 18:30:00.109726 14977 tablet_bootstrap.cc:654] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:00.110462 14977 log.cc:824] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:00.110529 14978 tablet_bootstrap.cc:654] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:00.111382 14981 tablet_bootstrap.cc:654] T 19f985963f3a4cceb6824b8e5b828a17 P fb02aaffab8341c5a7106692e01bcc1d: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:00.111392 14978 log.cc:824] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:00.112195 14981 log.cc:824] T 19f985963f3a4cceb6824b8e5b828a17 P fb02aaffab8341c5a7106692e01bcc1d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:00.112764 14977 tablet_bootstrap.cc:492] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b: No bootstrap required, opened a new log
I20260806 18:30:00.112952 14977 ts_tablet_manager.cc:1397] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:30:00.113364 14978 tablet_bootstrap.cc:492] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c: No bootstrap required, opened a new log
I20260806 18:30:00.113554 14978 ts_tablet_manager.cc:1397] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:30:00.113794 14981 tablet_bootstrap.cc:492] T 19f985963f3a4cceb6824b8e5b828a17 P fb02aaffab8341c5a7106692e01bcc1d: No bootstrap required, opened a new log
I20260806 18:30:00.113955 14981 ts_tablet_manager.cc:1397] T 19f985963f3a4cceb6824b8e5b828a17 P fb02aaffab8341c5a7106692e01bcc1d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:30:00.117586 14981 raft_consensus.cc:348] T 19f985963f3a4cceb6824b8e5b828a17 P fb02aaffab8341c5a7106692e01bcc1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506b453190034de99008c19eda99a34b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36557 } } peers { permanent_uuid: "261ed9e8e13c4d50b2277b60aa73c80c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39493 } } peers { permanent_uuid: "fb02aaffab8341c5a7106692e01bcc1d" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43019 } }
I20260806 18:30:00.117635 14977 raft_consensus.cc:348] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506b453190034de99008c19eda99a34b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36557 } } peers { permanent_uuid: "261ed9e8e13c4d50b2277b60aa73c80c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39493 } } peers { permanent_uuid: "fb02aaffab8341c5a7106692e01bcc1d" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43019 } }
I20260806 18:30:00.117635 14978 raft_consensus.cc:348] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506b453190034de99008c19eda99a34b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36557 } } peers { permanent_uuid: "261ed9e8e13c4d50b2277b60aa73c80c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39493 } } peers { permanent_uuid: "fb02aaffab8341c5a7106692e01bcc1d" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43019 } }
I20260806 18:30:00.117827 14981 raft_consensus.cc:374] T 19f985963f3a4cceb6824b8e5b828a17 P fb02aaffab8341c5a7106692e01bcc1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:00.117883 14981 raft_consensus.cc:729] T 19f985963f3a4cceb6824b8e5b828a17 P fb02aaffab8341c5a7106692e01bcc1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb02aaffab8341c5a7106692e01bcc1d, State: Initialized, Role: FOLLOWER
I20260806 18:30:00.117895 14977 raft_consensus.cc:374] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:00.117895 14978 raft_consensus.cc:374] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:00.117945 14978 raft_consensus.cc:729] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 261ed9e8e13c4d50b2277b60aa73c80c, State: Initialized, Role: FOLLOWER
I20260806 18:30:00.117957 14977 raft_consensus.cc:729] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506b453190034de99008c19eda99a34b, State: Initialized, Role: FOLLOWER
I20260806 18:30:00.118341 14981 consensus_queue.cc:260] T 19f985963f3a4cceb6824b8e5b828a17 P fb02aaffab8341c5a7106692e01bcc1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506b453190034de99008c19eda99a34b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36557 } } peers { permanent_uuid: "261ed9e8e13c4d50b2277b60aa73c80c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39493 } } peers { permanent_uuid: "fb02aaffab8341c5a7106692e01bcc1d" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43019 } }
I20260806 18:30:00.118435 14977 consensus_queue.cc:260] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506b453190034de99008c19eda99a34b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36557 } } peers { permanent_uuid: "261ed9e8e13c4d50b2277b60aa73c80c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39493 } } peers { permanent_uuid: "fb02aaffab8341c5a7106692e01bcc1d" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43019 } }
I20260806 18:30:00.118435 14978 consensus_queue.cc:260] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506b453190034de99008c19eda99a34b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36557 } } peers { permanent_uuid: "261ed9e8e13c4d50b2277b60aa73c80c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39493 } } peers { permanent_uuid: "fb02aaffab8341c5a7106692e01bcc1d" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43019 } }
I20260806 18:30:00.118664 14988 raft_consensus.cc:482] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:30:00.118873 14988 raft_consensus.cc:504] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506b453190034de99008c19eda99a34b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36557 } } peers { permanent_uuid: "261ed9e8e13c4d50b2277b60aa73c80c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39493 } } peers { permanent_uuid: "fb02aaffab8341c5a7106692e01bcc1d" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43019 } }
I20260806 18:30:00.119450 14977 ts_tablet_manager.cc:1428] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 18:30:00.119450 14978 ts_tablet_manager.cc:1428] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:30:00.119447 14981 ts_tablet_manager.cc:1428] T 19f985963f3a4cceb6824b8e5b828a17 P fb02aaffab8341c5a7106692e01bcc1d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:30:00.119812 14956 heartbeater.cc:503] Master 127.7.67.254:40373 was elected leader, sending a full tablet report...
I20260806 18:30:00.119813 14764 heartbeater.cc:503] Master 127.7.67.254:40373 was elected leader, sending a full tablet report...
I20260806 18:30:00.121117 14563 heartbeater.cc:503] Master 127.7.67.254:40373 was elected leader, sending a full tablet report...
I20260806 18:30:00.120920 14988 leader_election.cc:288] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 506b453190034de99008c19eda99a34b (127.7.67.195:36557), fb02aaffab8341c5a7106692e01bcc1d (127.7.67.193:43019)
I20260806 18:30:00.123581 14766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:00.120448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:36557 (local address 127.7.67.194:36849)
0806 18:30:00.120735 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:00.120751 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:00.120826 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:00.122392 (+ 1566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:00.122395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:00.122412 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:00.122998 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:30:00.123000 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:00.123012 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:30:00.123050 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:00.123341 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:00.123345 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:00.123348 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:30:00.123421 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 18:30:00.123459 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206}
I20260806 18:30:00.123895 14973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:00.120993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:36849 (local address 127.7.67.195:36557)
0806 18:30:00.122006 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:00.122010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:00.122027 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:30:00.122065 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:00.122068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:00.122100 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:00.122180 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:00.123158 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:00.123162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:00.123165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:00.123178 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:00.123203 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:00.123214 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:00.123294 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:00.123742 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 18:30:00.123783 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":919}
I20260806 18:30:00.124534 14893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f985963f3a4cceb6824b8e5b828a17" candidate_uuid: "261ed9e8e13c4d50b2277b60aa73c80c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506b453190034de99008c19eda99a34b" is_pre_election: true
I20260806 18:30:00.124804 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:00.120643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:34153 (local address 127.7.67.193:43019)
0806 18:30:00.120759 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:00.120764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:00.123382 (+ 2618us) server_negotiation.cc:407] Connection header received
0806 18:30:00.123487 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:00.123491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:00.123524 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:00.123597 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:00.124354 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:00.124357 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:00.124361 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:00.124373 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:00.124395 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:00.124405 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:00.124465 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:00.124649 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 18:30:00.124692 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:30:00.124989 14893 raft_consensus.cc:2454] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 261ed9e8e13c4d50b2277b60aa73c80c in term 0.
I20260806 18:30:00.125115 14992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:00.120461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:43019 (local address 127.7.67.194:34153)
0806 18:30:00.123302 (+ 2841us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:00.123316 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:00.123413 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:00.123695 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:00.123698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:00.123713 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:00.124183 (+ 470us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:30:00.124186 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:00.124197 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:30:00.124225 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:00.124545 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:00.124548 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:00.124551 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:30:00.124931 (+ 380us) client_negotiation.cc:240] Negotiation successful
0806 18:30:00.124968 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2775,"mutex_wait_us":225,"thread_start_us":113,"threads_started":1}
I20260806 18:30:00.125689 14589 leader_election.cc:302] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 261ed9e8e13c4d50b2277b60aa73c80c, 506b453190034de99008c19eda99a34b; no voters:
I20260806 18:30:00.125779 14496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f985963f3a4cceb6824b8e5b828a17" candidate_uuid: "261ed9e8e13c4d50b2277b60aa73c80c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb02aaffab8341c5a7106692e01bcc1d" is_pre_election: true
I20260806 18:30:00.126027 14496 raft_consensus.cc:2454] T 19f985963f3a4cceb6824b8e5b828a17 P fb02aaffab8341c5a7106692e01bcc1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 261ed9e8e13c4d50b2277b60aa73c80c in term 0.
I20260806 18:30:00.126194 14990 raft_consensus.cc:2781] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:30:00.126317 14990 raft_consensus.cc:482] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:30:00.126374 14990 raft_consensus.cc:3037] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:00.127561 14990 raft_consensus.cc:504] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506b453190034de99008c19eda99a34b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36557 } } peers { permanent_uuid: "261ed9e8e13c4d50b2277b60aa73c80c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39493 } } peers { permanent_uuid: "fb02aaffab8341c5a7106692e01bcc1d" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43019 } }
I20260806 18:30:00.127884 14990 leader_election.cc:288] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [CANDIDATE]: Term 1 election: Requested vote from peers 506b453190034de99008c19eda99a34b (127.7.67.195:36557), fb02aaffab8341c5a7106692e01bcc1d (127.7.67.193:43019)
I20260806 18:30:00.128221 14496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f985963f3a4cceb6824b8e5b828a17" candidate_uuid: "261ed9e8e13c4d50b2277b60aa73c80c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb02aaffab8341c5a7106692e01bcc1d"
I20260806 18:30:00.128224 14893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f985963f3a4cceb6824b8e5b828a17" candidate_uuid: "261ed9e8e13c4d50b2277b60aa73c80c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506b453190034de99008c19eda99a34b"
I20260806 18:30:00.128324 14496 raft_consensus.cc:3037] T 19f985963f3a4cceb6824b8e5b828a17 P fb02aaffab8341c5a7106692e01bcc1d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:00.128331 14893 raft_consensus.cc:3037] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:00.129472 14496 raft_consensus.cc:2454] T 19f985963f3a4cceb6824b8e5b828a17 P fb02aaffab8341c5a7106692e01bcc1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 261ed9e8e13c4d50b2277b60aa73c80c in term 1.
I20260806 18:30:00.129487 14893 raft_consensus.cc:2454] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 261ed9e8e13c4d50b2277b60aa73c80c in term 1.
I20260806 18:30:00.129767 14589 leader_election.cc:302] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 261ed9e8e13c4d50b2277b60aa73c80c, 506b453190034de99008c19eda99a34b; no voters:
I20260806 18:30:00.129933 14990 raft_consensus.cc:2781] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:00.130039 14990 raft_consensus.cc:686] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [term 1 LEADER]: Becoming Leader. State: Replica: 261ed9e8e13c4d50b2277b60aa73c80c, State: Running, Role: LEADER
I20260806 18:30:00.130344 14990 consensus_queue.cc:237] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506b453190034de99008c19eda99a34b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36557 } } peers { permanent_uuid: "261ed9e8e13c4d50b2277b60aa73c80c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39493 } } peers { permanent_uuid: "fb02aaffab8341c5a7106692e01bcc1d" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43019 } }
I20260806 18:30:00.133234 14277 catalog_manager.cc:5495] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c reported cstate change: term changed from 0 to 1, leader changed from <none> to 261ed9e8e13c4d50b2277b60aa73c80c (127.7.67.194). New cstate: current_term: 1 leader_uuid: "261ed9e8e13c4d50b2277b60aa73c80c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506b453190034de99008c19eda99a34b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "261ed9e8e13c4d50b2277b60aa73c80c" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb02aaffab8341c5a7106692e01bcc1d" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43019 } health_report { overall_health: UNKNOWN } } }
W20260806 18:30:00.134600 14765 tablet.cc:2369] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:30:00.143613 14975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:00.142294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52266 (local address 127.7.67.194:39493)
0806 18:30:00.142409 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:00.142413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:00.142427 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:30:00.142539 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:00.142542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:00.142597 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:00.142663 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:00.143276 (+ 613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:00.143278 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:00.143281 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:00.143291 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:00.143310 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:00.143317 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:00.143381 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:00.143433 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 18:30:00.143466 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:30:00.149379 14496 raft_consensus.cc:1261] T 19f985963f3a4cceb6824b8e5b828a17 P fb02aaffab8341c5a7106692e01bcc1d [term 1 FOLLOWER]: Refusing update from remote peer 261ed9e8e13c4d50b2277b60aa73c80c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:30:00.149386 14893 raft_consensus.cc:1261] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b [term 1 FOLLOWER]: Refusing update from remote peer 261ed9e8e13c4d50b2277b60aa73c80c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:30:00.150084 14988 consensus_queue.cc:1035] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb02aaffab8341c5a7106692e01bcc1d" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:30:00.150266 14990 consensus_queue.cc:1035] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [LEADER]: Connected to new peer: Peer: permanent_uuid: "506b453190034de99008c19eda99a34b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:30:00.157948 14999 mvcc.cc:205] Tried to move back new op lower bound from 7315623936604811264 to 7315623936537071616. Current Snapshot: MvccSnapshot[applied={T|T < 7315623936604811264}]
I20260806 18:30:00.160633 15005 mvcc.cc:205] Tried to move back new op lower bound from 7315623936604811264 to 7315623936537071616. Current Snapshot: MvccSnapshot[applied={T|T < 7315623936604811264}]
I20260806 18:30:00.167907 14277 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:30:00.168081 14277 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:30:00.170450 14277 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5239497d90ec43a8b3f67981559ef569: Sending DeleteTablet for 3 replicas of tablet 19f985963f3a4cceb6824b8e5b828a17
I20260806 18:30:00.171363 14658 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f985963f3a4cceb6824b8e5b828a17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:00 UTC) from {username='slave'} at 127.0.0.1:52260
I20260806 18:30:00.171597 14860 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f985963f3a4cceb6824b8e5b828a17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:00 UTC) from {username='slave'} at 127.0.0.1:35590
I20260806 18:30:00.171905 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 14367
I20260806 18:30:00.183161 15021 tablet_replica.cc:330] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b: stopping tablet replica
I20260806 18:30:00.183540 15021 raft_consensus.cc:2229] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:30:00.183923 15021 raft_consensus.cc:2258] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:00.183949 15020 tablet_replica.cc:330] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c: stopping tablet replica
I20260806 18:30:00.184275 15020 raft_consensus.cc:2229] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:30:00.184685 15020 raft_consensus.cc:2258] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:00.185883 15021 ts_tablet_manager.cc:1905] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:00.186103 15020 ts_tablet_manager.cc:1905] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:00.188800 15020 ts_tablet_manager.cc:1918] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:30:00.188800 15021 ts_tablet_manager.cc:1918] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:30:00.188910 15021 log.cc:1196] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/19f985963f3a4cceb6824b8e5b828a17
I20260806 18:30:00.188910 15020 log.cc:1196] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/19f985963f3a4cceb6824b8e5b828a17
I20260806 18:30:00.189227 15021 ts_tablet_manager.cc:1939] T 19f985963f3a4cceb6824b8e5b828a17 P 506b453190034de99008c19eda99a34b: Deleting consensus metadata
I20260806 18:30:00.189227 15020 ts_tablet_manager.cc:1939] T 19f985963f3a4cceb6824b8e5b828a17 P 261ed9e8e13c4d50b2277b60aa73c80c: Deleting consensus metadata
W20260806 18:30:00.189536 14264 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.193:43019 (error 108)
I20260806 18:30:00.190488 14263 catalog_manager.cc:4841] TS 261ed9e8e13c4d50b2277b60aa73c80c (127.7.67.194:39493): tablet 19f985963f3a4cceb6824b8e5b828a17 (table test-table [id=9d62752b66644a8fb3186c80a9ec316b]) successfully deleted
I20260806 18:30:00.190510 14262 catalog_manager.cc:4841] TS 506b453190034de99008c19eda99a34b (127.7.67.195:36557): tablet 19f985963f3a4cceb6824b8e5b828a17 (table test-table [id=9d62752b66644a8fb3186c80a9ec316b]) successfully deleted
I20260806 18:30:00.190950 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 14569
I20260806 18:30:00.195536 14972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:00.193800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:43019 (local address 127.0.0.1:56110)
0806 18:30:00.195347 (+ 1547us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:00.195429 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:43019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1442}
W20260806 18:30:00.196658 14264 catalog_manager.cc:4584] TS fb02aaffab8341c5a7106692e01bcc1d (127.7.67.193:43019): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 19f985963f3a4cceb6824b8e5b828a17: Network error: Client connection negotiation failed: client connection to 127.7.67.193:43019: connect: Connection refused (error 111)
I20260806 18:30:00.202731 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 14767
I20260806 18:30:00.216873 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 14236
2026-08-06T18:30:00Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (940 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 18:30:00.245070 7439 mini_oidc.cc:152] Starting JWKS server
I20260806 18:30:00.246812 7439 webserver.cc:460] Webserver started at https://127.0.0.1:36963/ using document root <none> and password file <none>
I20260806 18:30:00.247021 7439 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 18:30:00.247293 7439 webserver.cc:460] Webserver started at http://127.0.0.1:46001/ using document root <none> and password file <none>
2026-08-06T18:30:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:30:00Z Disabled control of system clock
I20260806 18:30:00.257464 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34301
--webserver_interface=127.7.67.254
--webserver_port=0
--builtin_ntp_servers=127.7.67.212:34975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:34301
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36963/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0/testchainca.cert with env {}
W20260806 18:30:00.373754 15030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:00.374068 15030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:00.374118 15030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:00.378257 15030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:30:00.378342 15030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:00.378360 15030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:00.378379 15030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:30:00.378397 15030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:30:00.378415 15030 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:30:00.378439 15030 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36963/jwks.json
I20260806 18:30:00.383941 15030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:34975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:34301
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36963/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:00.385017 15030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:00.386027 15030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:00.394649 15036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:00.394892 15042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:00.394913 15035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:00.395017 15030 server_base.cc:1034] running on GCE node
I20260806 18:30:00.395709 15030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:00.396812 15030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:00.397990 15030 hybrid_clock.cc:648] HybridClock initialized: now 1786041000397969 us; error 32 us; skew 500 ppm
I20260806 18:30:00.399883 15030 webserver.cc:460] Webserver started at http://127.7.67.254:37617/ using document root <none> and password file <none>
I20260806 18:30:00.400437 15030 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:00.400497 15030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:00.400671 15030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:00.402796 15030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "ed07887bafc44c6d8cf973b7d2812308"
format_stamp: "Formatted at 2026-08-06 18:30:00 on dist-test-slave-5bn2"
server_key: "d9fb82f1ac9125021a65badda0af3281"
server_key_iv: "e7536317579d593812ed5ab7addc8af8"
server_key_version: "encryptionkey@0"
I20260806 18:30:00.403441 15030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "ed07887bafc44c6d8cf973b7d2812308"
format_stamp: "Formatted at 2026-08-06 18:30:00 on dist-test-slave-5bn2"
server_key: "d9fb82f1ac9125021a65badda0af3281"
server_key_iv: "e7536317579d593812ed5ab7addc8af8"
server_key_version: "encryptionkey@0"
I20260806 18:30:00.407295 15030 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 18:30:00.409838 15052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:00.411161 15030 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:30:00.411278 15030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "ed07887bafc44c6d8cf973b7d2812308"
format_stamp: "Formatted at 2026-08-06 18:30:00 on dist-test-slave-5bn2"
server_key: "d9fb82f1ac9125021a65badda0af3281"
server_key_iv: "e7536317579d593812ed5ab7addc8af8"
server_key_version: "encryptionkey@0"
I20260806 18:30:00.411381 15030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:00.426908 15030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:00.427634 15030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:00.427805 15030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:00.435232 15030 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:34301
I20260806 18:30:00.435251 15123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:34301 every 8 connection(s)
I20260806 18:30:00.436236 15030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:30:00.439553 15124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:00.443989 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 15030
I20260806 18:30:00.444101 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:30:00.444366 7439 external_mini_cluster.cc:1428] Setting key f3d1a8db86bb0f28304f90f78a8518ab
I20260806 18:30:00.445432 15124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308: Bootstrap starting.
I20260806 18:30:00.447856 15124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:00.448747 15124 log.cc:824] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:00.450652 15124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308: No bootstrap required, opened a new log
I20260806 18:30:00.453397 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:00.445805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60882 (local address 127.7.67.254:34301)
0806 18:30:00.446444 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:00.446458 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:00.446494 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:30:00.447188 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:00.447208 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:00.447637 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:00.447923 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:00.448309 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:00.449301 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:00.450196 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:00.450659 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:00.450812 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:00.451637 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:00.451642 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:00.451655 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:00.452054 (+ 399us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:00.452112 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:00.452133 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:00.452211 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:00.452358 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 18:30:00.452622 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":123,"threads_started":1}
I20260806 18:30:00.454653 15124 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed07887bafc44c6d8cf973b7d2812308" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34301 } }
I20260806 18:30:00.454967 15124 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:00.455029 15124 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed07887bafc44c6d8cf973b7d2812308, State: Initialized, Role: FOLLOWER
I20260806 18:30:00.455488 15124 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed07887bafc44c6d8cf973b7d2812308" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34301 } }
I20260806 18:30:00.455664 15124 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:30:00.455734 15124 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:30:00.455837 15124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:00.456399 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:30:00.457024 15124 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed07887bafc44c6d8cf973b7d2812308" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34301 } }
I20260806 18:30:00.457396 15124 leader_election.cc:302] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed07887bafc44c6d8cf973b7d2812308; no voters:
I20260806 18:30:00.457717 15124 leader_election.cc:288] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:30:00.457823 15129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:00.458014 15129 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308 [term 1 LEADER]: Becoming Leader. State: Replica: ed07887bafc44c6d8cf973b7d2812308, State: Running, Role: LEADER
I20260806 18:30:00.458318 15129 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed07887bafc44c6d8cf973b7d2812308" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34301 } }
I20260806 18:30:00.459100 15124 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:30:00.459614 15129 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed07887bafc44c6d8cf973b7d2812308. Latest consensus state: current_term: 1 leader_uuid: "ed07887bafc44c6d8cf973b7d2812308" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed07887bafc44c6d8cf973b7d2812308" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34301 } } }
I20260806 18:30:00.459736 15129 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:00.460156 15130 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed07887bafc44c6d8cf973b7d2812308" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed07887bafc44c6d8cf973b7d2812308" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34301 } } }
I20260806 18:30:00.460263 15130 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:00.460558 15134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:30:00.464236 15134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:30:00.469782 15134 catalog_manager.cc:1347] Generated new cluster ID: 876ba807fe8146f6b3628ea066ac5e9d
I20260806 18:30:00.469862 15134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:30:00.505129 15134 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:30:00.506115 15134 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:30:00.516247 15134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed07887bafc44c6d8cf973b7d2812308: Generated new TSK 0
I20260806 18:30:00.516981 15134 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 18:30:00.556512 15168 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 18:30:00.556663 15168 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 18:30:00.557353 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:00.539831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60892 (local address 127.7.67.254:34301)
0806 18:30:00.539968 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:00.539973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:00.540136 (+ 163us) server_negotiation.cc:407] Connection header received
0806 18:30:00.540349 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:00.540353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:00.540415 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:00.540540 (+ 125us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:30:00.540765 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:00.541596 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:00.542426 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:00.542787 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:00.542919 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:00.543253 (+ 334us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:30:00.556890 (+ 13637us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36963/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 18:30:00.557197 (+ 307us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60892: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36963/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 18:30:00.557443 15127 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60892: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36963/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 18:30:00.558696 15167 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:00.539721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34301 (local address 127.0.0.1:60892)
0806 18:30:00.540054 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:00.540067 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:00.540191 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:00.540543 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:00.540545 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:00.540551 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:30:00.540667 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:00.540671 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:00.541706 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:00.541708 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:00.542327 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:00.542334 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:00.542924 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:00.542928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:00.543007 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:00.543057 (+ 50us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:30:00.557134 (+ 14077us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36963/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 18:30:00.557352 (+ 218us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.67.254:34301: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36963/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":104,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.67.254:34301: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36963/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 18:30:00.559590 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 15030
2026-08-06T18:30:00Z chronyd exiting
I20260806 18:30:00.604190 7439 test_util.cc:175] -----------------------------------------------
I20260806 18:30:00.604290 7439 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786040973123132-7439-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (360 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 18:30:00.605468 7439 mini_oidc.cc:152] Starting JWKS server
I20260806 18:30:00.606333 7439 webserver.cc:460] Webserver started at https://127.0.0.1:45139/ using document root <none> and password file <none>
I20260806 18:30:00.606434 7439 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 18:30:00.606683 7439 webserver.cc:460] Webserver started at http://127.0.0.1:41707/ using document root <none> and password file <none>
2026-08-06T18:30:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:30:00Z Disabled control of system clock
I20260806 18:30:00.616842 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34995
--webserver_interface=127.7.67.254
--webserver_port=0
--builtin_ntp_servers=127.7.67.212:37675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:34995
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45139/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786040973123132-7439-0/testchainca.cert with env {}
W20260806 18:30:00.729364 15177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:00.729663 15177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:00.729723 15177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:00.733821 15177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:30:00.733893 15177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:00.733914 15177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:00.733935 15177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:30:00.733954 15177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:30:00.733981 15177 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:30:00.734009 15177 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45139/jwks.json
I20260806 18:30:00.739569 15177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:37675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:34995
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45139/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786040973123132-7439-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:00.740707 15177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:00.741770 15177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:00.750891 15187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:00.750754 15191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:00.750754 15186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:00.751449 15177 server_base.cc:1034] running on GCE node
I20260806 18:30:00.752157 15177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:00.753316 15177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:00.754474 15177 hybrid_clock.cc:648] HybridClock initialized: now 1786041000754465 us; error 38 us; skew 500 ppm
I20260806 18:30:00.756428 15177 webserver.cc:460] Webserver started at http://127.7.67.254:37031/ using document root <none> and password file <none>
I20260806 18:30:00.757030 15177 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:00.757102 15177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:00.757301 15177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:00.759469 15177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "9f01f9d3853e4434818887b0860a2a6b"
format_stamp: "Formatted at 2026-08-06 18:30:00 on dist-test-slave-5bn2"
server_key: "973b9e304565bab8904f5b3b03569db1"
server_key_iv: "94a13e22c27339614c8219482df074b6"
server_key_version: "encryptionkey@0"
I20260806 18:30:00.760054 15177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "9f01f9d3853e4434818887b0860a2a6b"
format_stamp: "Formatted at 2026-08-06 18:30:00 on dist-test-slave-5bn2"
server_key: "973b9e304565bab8904f5b3b03569db1"
server_key_iv: "94a13e22c27339614c8219482df074b6"
server_key_version: "encryptionkey@0"
I20260806 18:30:00.763960 15177 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:30:00.766396 15198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:00.767627 15177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:30:00.767762 15177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "9f01f9d3853e4434818887b0860a2a6b"
format_stamp: "Formatted at 2026-08-06 18:30:00 on dist-test-slave-5bn2"
server_key: "973b9e304565bab8904f5b3b03569db1"
server_key_iv: "94a13e22c27339614c8219482df074b6"
server_key_version: "encryptionkey@0"
I20260806 18:30:00.767868 15177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:00.795938 15177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:00.796708 15177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:00.796898 15177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:00.803829 15177 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:34995
I20260806 18:30:00.803841 15269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:34995 every 8 connection(s)
I20260806 18:30:00.804808 15177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:30:00.808056 15270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:00.813261 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 15177
I20260806 18:30:00.813387 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:30:00.813668 7439 external_mini_cluster.cc:1428] Setting key bd11b41a6f4f9092ba657111297cb79b
I20260806 18:30:00.813948 15270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b: Bootstrap starting.
I20260806 18:30:00.816751 15270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:00.817685 15270 log.cc:824] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:00.819943 15270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b: No bootstrap required, opened a new log
I20260806 18:30:00.824385 15274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:00.815127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56486 (local address 127.7.67.254:34995)
0806 18:30:00.815843 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:00.815862 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:00.815904 (+ 42us) server_negotiation.cc:407] Connection header received
0806 18:30:00.816752 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:00.816777 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:00.817342 (+ 565us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:00.817712 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:00.818184 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:00.819558 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:00.820478 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:00.821176 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:00.821338 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:00.822123 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:00.822130 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:00.822146 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:00.822660 (+ 514us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:00.822733 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:00.822759 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:00.822896 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:00.823037 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 18:30:00.823348 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":129,"threads_started":1}
I20260806 18:30:00.824574 15270 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f01f9d3853e4434818887b0860a2a6b" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34995 } }
I20260806 18:30:00.824838 15270 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:00.824906 15270 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f01f9d3853e4434818887b0860a2a6b, State: Initialized, Role: FOLLOWER
I20260806 18:30:00.825331 15270 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f01f9d3853e4434818887b0860a2a6b" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34995 } }
I20260806 18:30:00.825477 15270 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:30:00.825558 15270 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:30:00.825654 15270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:00.826766 15270 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f01f9d3853e4434818887b0860a2a6b" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34995 } }
I20260806 18:30:00.827140 15270 leader_election.cc:302] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f01f9d3853e4434818887b0860a2a6b; no voters:
I20260806 18:30:00.827450 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:30:00.827461 15270 leader_election.cc:288] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:30:00.827543 15276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:00.827718 15276 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b [term 1 LEADER]: Becoming Leader. State: Replica: 9f01f9d3853e4434818887b0860a2a6b, State: Running, Role: LEADER
I20260806 18:30:00.828023 15276 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f01f9d3853e4434818887b0860a2a6b" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34995 } }
I20260806 18:30:00.828804 15270 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:30:00.829824 15277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f01f9d3853e4434818887b0860a2a6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f01f9d3853e4434818887b0860a2a6b" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34995 } } }
I20260806 18:30:00.829823 15278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f01f9d3853e4434818887b0860a2a6b. Latest consensus state: current_term: 1 leader_uuid: "9f01f9d3853e4434818887b0860a2a6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f01f9d3853e4434818887b0860a2a6b" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34995 } } }
I20260806 18:30:00.829977 15278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:00.829977 15277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:00.830410 15290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:30:00.833583 15290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:30:00.839311 15290 catalog_manager.cc:1347] Generated new cluster ID: 8a9d42f54bb946d585d04586fc263e0d
I20260806 18:30:00.839388 15290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:30:00.850754 15290 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:30:00.851713 15290 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:30:00.862994 15290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f01f9d3853e4434818887b0860a2a6b: Generated new TSK 0
I20260806 18:30:00.863880 15290 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 18:30:00.902634 15311 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 18:30:00.902740 15311 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 18:30:00.903425 15274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:00.880373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56490 (local address 127.7.67.254:34995)
0806 18:30:00.880504 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:00.880509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:00.880651 (+ 142us) server_negotiation.cc:407] Connection header received
0806 18:30:00.880816 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:00.880819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:00.880867 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:00.880955 (+ 88us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:30:00.881191 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:00.881723 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:00.882943 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:00.883170 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:00.883245 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:00.883547 (+ 302us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:30:00.903031 (+ 19484us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45139/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 18:30:00.903295 (+ 264us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56490: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45139/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 18:30:00.903497 15274 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56490: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45139/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 18:30:00.903667 15310 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:00.880254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34995 (local address 127.0.0.1:56490)
0806 18:30:00.880559 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:00.880574 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:00.880696 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:00.880969 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:00.880972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:00.880980 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:30:00.881103 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:00.881109 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:00.881817 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:00.881820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:00.882816 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:00.882826 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:00.883270 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:00.883274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:00.883363 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:00.883394 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:30:00.903261 (+ 19867us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45139/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 18:30:00.903477 (+ 216us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.67.254:34995: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45139/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":104,"threads_started":1}
I20260806 18:30:00.904412 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 15177
2026-08-06T18:30:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (345 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 18:30:00.951082 7439 mini_oidc.cc:152] Starting JWKS server
I20260806 18:30:00.951968 7439 webserver.cc:460] Webserver started at https://127.0.0.1:34937/ using document root <none> and password file <none>
I20260806 18:30:00.952069 7439 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 18:30:00.952312 7439 webserver.cc:460] Webserver started at http://127.0.0.1:43231/ using document root <none> and password file <none>
2026-08-06T18:30:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:30:00Z Disabled control of system clock
I20260806 18:30:00.962312 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34341
--webserver_interface=127.7.67.254
--webserver_port=0
--builtin_ntp_servers=127.7.67.212:34267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:34341
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34937/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 18:30:01.076467 15323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:01.076776 15323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:01.076817 15323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:01.080964 15323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:30:01.081041 15323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:01.081055 15323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:01.081077 15323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:30:01.081091 15323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:30:01.081105 15323 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:30:01.081117 15323 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34937/jwks.json
I20260806 18:30:01.085999 15323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:34267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:34341
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34937/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:01.087256 15323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:01.088743 15323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:01.097352 15331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:01.097361 15330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:01.097582 15323 server_base.cc:1034] running on GCE node
W20260806 18:30:01.097352 15335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:01.098392 15323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:01.099608 15323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:01.100768 15323 hybrid_clock.cc:648] HybridClock initialized: now 1786041001100745 us; error 39 us; skew 500 ppm
I20260806 18:30:01.102725 15323 webserver.cc:460] Webserver started at http://127.7.67.254:44411/ using document root <none> and password file <none>
I20260806 18:30:01.103348 15323 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:01.103416 15323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:01.103602 15323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:01.105749 15323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "259851ea46e845beaf2165038783452f"
format_stamp: "Formatted at 2026-08-06 18:30:01 on dist-test-slave-5bn2"
server_key: "bb6acee47e5c7b08c18b171a8faed51d"
server_key_iv: "7f02cdc2f80576c3b33a5d4dab33dd57"
server_key_version: "encryptionkey@0"
I20260806 18:30:01.106305 15323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "259851ea46e845beaf2165038783452f"
format_stamp: "Formatted at 2026-08-06 18:30:01 on dist-test-slave-5bn2"
server_key: "bb6acee47e5c7b08c18b171a8faed51d"
server_key_iv: "7f02cdc2f80576c3b33a5d4dab33dd57"
server_key_version: "encryptionkey@0"
I20260806 18:30:01.110178 15323 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:30:01.112663 15342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:01.113893 15323 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:30:01.114002 15323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "259851ea46e845beaf2165038783452f"
format_stamp: "Formatted at 2026-08-06 18:30:01 on dist-test-slave-5bn2"
server_key: "bb6acee47e5c7b08c18b171a8faed51d"
server_key_iv: "7f02cdc2f80576c3b33a5d4dab33dd57"
server_key_version: "encryptionkey@0"
I20260806 18:30:01.114099 15323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:01.128364 15323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:01.129046 15323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:01.129204 15323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:01.136336 15323 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:34341
I20260806 18:30:01.136351 15414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:34341 every 8 connection(s)
I20260806 18:30:01.137321 15323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:30:01.138340 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 15323
I20260806 18:30:01.138464 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:30:01.138774 7439 external_mini_cluster.cc:1428] Setting key 9140e4ce54765122eba13d30a584ff37
I20260806 18:30:01.141186 15417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:01.146839 15417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f: Bootstrap starting.
I20260806 18:30:01.148058 15422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:01.140240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59736 (local address 127.7.67.254:34341)
0806 18:30:01.140931 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:01.140947 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:01.140983 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:30:01.141693 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:01.141712 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:01.142114 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:01.142427 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:01.142907 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:01.143927 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:01.144828 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:01.145290 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:01.145438 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:01.146327 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:01.146332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:01.146345 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:01.146733 (+ 388us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:01.146791 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:01.146811 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:01.146933 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:01.147117 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 18:30:01.147358 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":147,"threads_started":1}
I20260806 18:30:01.149153 15417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:01.149933 15417 log.cc:824] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:01.150821 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:30:01.152069 15417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f: No bootstrap required, opened a new log
I20260806 18:30:01.155893 15417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259851ea46e845beaf2165038783452f" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34341 } }
I20260806 18:30:01.156124 15417 raft_consensus.cc:374] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:01.156176 15417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 259851ea46e845beaf2165038783452f, State: Initialized, Role: FOLLOWER
I20260806 18:30:01.156591 15417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259851ea46e845beaf2165038783452f" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34341 } }
I20260806 18:30:01.156713 15417 raft_consensus.cc:388] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:30:01.156772 15417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:30:01.156862 15417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:01.157771 15417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259851ea46e845beaf2165038783452f" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34341 } }
I20260806 18:30:01.158092 15417 leader_election.cc:302] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 259851ea46e845beaf2165038783452f; no voters:
I20260806 18:30:01.158387 15417 leader_election.cc:288] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:30:01.158529 15425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:01.158802 15425 raft_consensus.cc:686] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f [term 1 LEADER]: Becoming Leader. State: Replica: 259851ea46e845beaf2165038783452f, State: Running, Role: LEADER
I20260806 18:30:01.159183 15425 consensus_queue.cc:237] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259851ea46e845beaf2165038783452f" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34341 } }
I20260806 18:30:01.159693 15417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:30:01.160818 15426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "259851ea46e845beaf2165038783452f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259851ea46e845beaf2165038783452f" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34341 } } }
I20260806 18:30:01.160874 15428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 259851ea46e845beaf2165038783452f. Latest consensus state: current_term: 1 leader_uuid: "259851ea46e845beaf2165038783452f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259851ea46e845beaf2165038783452f" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34341 } } }
I20260806 18:30:01.160993 15426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:01.161013 15428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:01.161510 15439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:30:01.164297 15439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:30:01.169461 15439 catalog_manager.cc:1347] Generated new cluster ID: 9986d487f6064c9189759d9fa7771f10
I20260806 18:30:01.169531 15439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:30:01.183638 15439 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:30:01.184602 15439 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:30:01.191397 15439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 259851ea46e845beaf2165038783452f: Generated new TSK 0
I20260806 18:30:01.192162 15439 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:30:01.214290 15422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:01.209768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59744 (local address 127.7.67.254:34341)
0806 18:30:01.209896 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:01.209901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:01.210046 (+ 145us) server_negotiation.cc:407] Connection header received
0806 18:30:01.210248 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:01.210251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:01.210300 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:01.210390 (+ 90us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:30:01.210629 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:01.211205 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:01.212059 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:01.212270 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:01.212359 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:01.212683 (+ 324us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:30:01.213913 (+ 1230us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34937/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 18:30:01.214163 (+ 250us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59744: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34937/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 18:30:01.214365 15422 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59744: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34937/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 18:30:01.217178 15457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:01.209676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34341 (local address 127.0.0.1:59744)
0806 18:30:01.209966 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:01.209978 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:01.210109 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:01.210403 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:01.210406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:01.210414 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:30:01.210522 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:01.210528 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:01.211323 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:01.211328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:01.211946 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:01.211954 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:01.212382 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:01.212386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:01.212486 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:01.212510 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:30:01.214112 (+ 1602us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34937/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 18:30:01.217012 (+ 2900us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.67.254:34341: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34937/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":2675,"thread_start_us":106,"threads_started":1}
I20260806 18:30:01.217904 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 15323
E20260806 18:30:01.245513 15459 webserver.cc:533] Webserver: sslize failed:
E20260806 18:30:01.245615 15459 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T18:30:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (326 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T18:30:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:30:01Z Disabled control of system clock
I20260806 18:30:01.287396 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34303
--webserver_interface=127.7.67.254
--webserver_port=0
--builtin_ntp_servers=127.7.67.212:42791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:34303
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786040973123132-7439-0/testchainca.cert with env {}
W20260806 18:30:01.403918 15464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:01.404237 15464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:01.404294 15464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:01.408353 15464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:30:01.408425 15464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:01.408447 15464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:01.408469 15464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:30:01.408490 15464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:30:01.408519 15464 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:30:01.408540 15464 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 18:30:01.414121 15464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:34303
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786040973123132-7439-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:01.415362 15464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:01.416414 15464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:01.425154 15477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:01.425154 15473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:01.425175 15475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:01.425477 15464 server_base.cc:1034] running on GCE node
I20260806 18:30:01.426112 15464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:01.427402 15464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:01.428577 15464 hybrid_clock.cc:648] HybridClock initialized: now 1786041001428551 us; error 40 us; skew 500 ppm
I20260806 18:30:01.430622 15464 webserver.cc:460] Webserver started at http://127.7.67.254:38779/ using document root <none> and password file <none>
I20260806 18:30:01.431249 15464 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:01.431318 15464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:01.431524 15464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:01.433614 15464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "a64519bbf0534ef8a94d58b326a77a08"
format_stamp: "Formatted at 2026-08-06 18:30:01 on dist-test-slave-5bn2"
server_key: "2ec6df16ea61b39fc2a807557e9546a7"
server_key_iv: "d8bf1d12530cd316583c2e14d15a0c95"
server_key_version: "encryptionkey@0"
I20260806 18:30:01.434165 15464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "a64519bbf0534ef8a94d58b326a77a08"
format_stamp: "Formatted at 2026-08-06 18:30:01 on dist-test-slave-5bn2"
server_key: "2ec6df16ea61b39fc2a807557e9546a7"
server_key_iv: "d8bf1d12530cd316583c2e14d15a0c95"
server_key_version: "encryptionkey@0"
I20260806 18:30:01.438000 15464 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:30:01.440577 15483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:01.441880 15464 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:30:01.442008 15464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "a64519bbf0534ef8a94d58b326a77a08"
format_stamp: "Formatted at 2026-08-06 18:30:01 on dist-test-slave-5bn2"
server_key: "2ec6df16ea61b39fc2a807557e9546a7"
server_key_iv: "d8bf1d12530cd316583c2e14d15a0c95"
server_key_version: "encryptionkey@0"
I20260806 18:30:01.442118 15464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:01.461750 15464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:01.462514 15464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:01.462703 15464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:01.469725 15464 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:34303
I20260806 18:30:01.469743 15555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:34303 every 8 connection(s)
I20260806 18:30:01.470726 15464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:30:01.473482 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 15464
I20260806 18:30:01.473601 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:30:01.473676 15556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:01.473888 7439 external_mini_cluster.cc:1428] Setting key 04ecf53cc04b99b5e8822d7f54bf6c8d
I20260806 18:30:01.480229 15556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08: Bootstrap starting.
I20260806 18:30:01.482625 15556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:01.483289 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:01.475269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34594 (local address 127.7.67.254:34303)
0806 18:30:01.475953 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:01.475970 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:01.476007 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:30:01.476920 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:01.476940 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:01.477351 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:01.477668 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:01.478107 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:01.479114 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:01.480008 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:01.480516 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:01.480667 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:01.481499 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:01.481504 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:01.481518 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:01.481937 (+ 419us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:01.481998 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:01.482019 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:01.482105 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:01.482236 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 18:30:01.482499 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":140,"threads_started":1}
I20260806 18:30:01.483438 15556 log.cc:824] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:01.485452 15556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08: No bootstrap required, opened a new log
I20260806 18:30:01.486193 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:30:01.489539 15556 raft_consensus.cc:348] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a64519bbf0534ef8a94d58b326a77a08" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34303 } }
I20260806 18:30:01.489781 15556 raft_consensus.cc:374] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:01.489840 15556 raft_consensus.cc:729] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a64519bbf0534ef8a94d58b326a77a08, State: Initialized, Role: FOLLOWER
I20260806 18:30:01.490269 15556 consensus_queue.cc:260] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a64519bbf0534ef8a94d58b326a77a08" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34303 } }
I20260806 18:30:01.490399 15556 raft_consensus.cc:388] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:30:01.490468 15556 raft_consensus.cc:482] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:30:01.490566 15556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:01.491568 15556 raft_consensus.cc:504] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a64519bbf0534ef8a94d58b326a77a08" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34303 } }
I20260806 18:30:01.491895 15556 leader_election.cc:302] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a64519bbf0534ef8a94d58b326a77a08; no voters:
I20260806 18:30:01.492249 15556 leader_election.cc:288] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:30:01.492343 15563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:01.492589 15563 raft_consensus.cc:686] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08 [term 1 LEADER]: Becoming Leader. State: Replica: a64519bbf0534ef8a94d58b326a77a08, State: Running, Role: LEADER
I20260806 18:30:01.492915 15563 consensus_queue.cc:237] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a64519bbf0534ef8a94d58b326a77a08" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34303 } }
I20260806 18:30:01.493568 15556 sys_catalog.cc:564] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:30:01.494589 15565 sys_catalog.cc:455] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a64519bbf0534ef8a94d58b326a77a08. Latest consensus state: current_term: 1 leader_uuid: "a64519bbf0534ef8a94d58b326a77a08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a64519bbf0534ef8a94d58b326a77a08" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34303 } } }
I20260806 18:30:01.494724 15565 sys_catalog.cc:458] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:01.495016 15564 sys_catalog.cc:455] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a64519bbf0534ef8a94d58b326a77a08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a64519bbf0534ef8a94d58b326a77a08" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34303 } } }
I20260806 18:30:01.495123 15575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:30:01.495147 15564 sys_catalog.cc:458] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:01.498440 15575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:30:01.504362 15575 catalog_manager.cc:1347] Generated new cluster ID: d4f078c27c7e4a6cb249446673e2125b
I20260806 18:30:01.504441 15575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:30:01.525668 15575 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:30:01.526661 15575 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:30:01.532632 15575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a64519bbf0534ef8a94d58b326a77a08: Generated new TSK 0
I20260806 18:30:01.533313 15575 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:30:02.017256 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:01.545286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34608 (local address 127.7.67.254:34303)
0806 18:30:01.545433 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:01.545438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:01.545588 (+ 150us) server_negotiation.cc:407] Connection header received
0806 18:30:01.545767 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:01.545771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:01.545831 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:01.545941 (+ 110us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:30:01.546163 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:01.547058 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:01.547890 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:01.548244 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:01.548343 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:01.548625 (+ 282us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:30:02.016806 (+468181us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 18:30:02.017115 (+ 309us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34608: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 18:30:02.017340 15560 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34608: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 18:30:02.017499 15598 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:01.545186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34303 (local address 127.0.0.1:34608)
0806 18:30:01.545507 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:01.545520 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:01.545628 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:01.545960 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:01.545962 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:01.545969 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:30:01.546072 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:01.546077 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:01.547167 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:01.547169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:01.547789 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:01.547797 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:01.548346 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:01.548348 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:01.548435 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:01.548459 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:30:02.017070 (+468611us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 18:30:02.017264 (+ 194us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.67.254:34303: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":102,"threads_started":1}
I20260806 18:30:02.018075 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 15464
2026-08-06T18:30:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (766 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (128 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (126 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:30:02 dist-test-slave-5bn2 krb5kdc[15612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:30:02 dist-test-slave-5bn2 krb5kdc[15612](info): set up 2 sockets
Aug 06 18:30:02 dist-test-slave-5bn2 krb5kdc[15612](info): commencing operation
krb5kdc: starting...
W20260806 18:30:03.408272 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:30:03 dist-test-slave-5bn2 krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:30:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:30:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:30:03.528903 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:40455
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:39881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:40455
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:03.645118 15629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:03.645411 15629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:03.645473 15629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:03.649530 15629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:30:03.649602 15629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:03.649628 15629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:03.649648 15629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:30:03.649667 15629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:30:03.655064 15629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:39881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:40455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:40455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:03.656139 15629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:03.657173 15629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:03.665725 15636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:03.665719 15635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:03.665839 15629 server_base.cc:1034] running on GCE node
W20260806 18:30:03.665719 15639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:03.666533 15629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:03.667695 15629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:03.668845 15629 hybrid_clock.cc:648] HybridClock initialized: now 1786041003668823 us; error 38 us; skew 500 ppm
Aug 06 18:30:03 dist-test-slave-5bn2 krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041003, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:03.674427 15629 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:30:03.675437 15629 webserver.cc:460] Webserver started at http://127.7.67.254:39217/ using document root <none> and password file <none>
I20260806 18:30:03.676005 15629 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:03.676069 15629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:03.676267 15629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:03.678333 15629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "7ae918f8f30249289e62d6a23115299a"
format_stamp: "Formatted at 2026-08-06 18:30:03 on dist-test-slave-5bn2"
server_key: "2682fb45828e76b67ba4c3322c6df369"
server_key_iv: "2494b43ff24b252c0682791891aabfca"
server_key_version: "encryptionkey@0"
I20260806 18:30:03.678932 15629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "7ae918f8f30249289e62d6a23115299a"
format_stamp: "Formatted at 2026-08-06 18:30:03 on dist-test-slave-5bn2"
server_key: "2682fb45828e76b67ba4c3322c6df369"
server_key_iv: "2494b43ff24b252c0682791891aabfca"
server_key_version: "encryptionkey@0"
I20260806 18:30:03.682591 15629 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:30:03.685170 15650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:03.686445 15629 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:30:03.686559 15629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "7ae918f8f30249289e62d6a23115299a"
format_stamp: "Formatted at 2026-08-06 18:30:03 on dist-test-slave-5bn2"
server_key: "2682fb45828e76b67ba4c3322c6df369"
server_key_iv: "2494b43ff24b252c0682791891aabfca"
server_key_version: "encryptionkey@0"
I20260806 18:30:03.686671 15629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:03.706090 15629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:03.709543 15629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:03.709748 15629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:03.716976 15629 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:40455
I20260806 18:30:03.716991 15730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:40455 every 8 connection(s)
I20260806 18:30:03.717980 15629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:30:03.720952 15731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:03.725649 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 15629
I20260806 18:30:03.725773 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:30:03.726047 7439 external_mini_cluster.cc:1428] Setting key 0ca8d16fa8a45c9c518ee9180647d943
I20260806 18:30:03.726917 15731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a: Bootstrap starting.
I20260806 18:30:03.729701 15731 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:03.730635 15731 log.cc:824] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:03.732769 15731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a: No bootstrap required, opened a new log
Aug 06 18:30:03 dist-test-slave-5bn2 krb5kdc[15612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:30:03.736959 15731 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ae918f8f30249289e62d6a23115299a" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40455 } }
I20260806 18:30:03.737273 15731 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:03.737344 15731 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ae918f8f30249289e62d6a23115299a, State: Initialized, Role: FOLLOWER
I20260806 18:30:03.737865 15731 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ae918f8f30249289e62d6a23115299a" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40455 } }
I20260806 18:30:03.738035 15731 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:30:03.738106 15731 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:30:03.738209 15731 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:03.739415 15731 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ae918f8f30249289e62d6a23115299a" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40455 } }
I20260806 18:30:03.739805 15731 leader_election.cc:302] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ae918f8f30249289e62d6a23115299a; no voters:
I20260806 18:30:03.740170 15731 leader_election.cc:288] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:30:03.740267 15736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:03.740509 15736 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a [term 1 LEADER]: Becoming Leader. State: Replica: 7ae918f8f30249289e62d6a23115299a, State: Running, Role: LEADER
I20260806 18:30:03.740875 15736 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ae918f8f30249289e62d6a23115299a" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40455 } }
I20260806 18:30:03.741159 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:03.727500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38820 (local address 127.7.67.254:40455)
0806 18:30:03.727951 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:03.727972 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:03.728016 (+ 44us) server_negotiation.cc:407] Connection header received
0806 18:30:03.728868 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:03.728892 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:03.729315 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:03.729686 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:03.730406 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:03.731740 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:03.732623 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:03.733302 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:03.733465 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:03.735904 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:03.735910 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:03.735930 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:03.735964 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:03.737853 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:03.738317 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:03.738324 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:03.738330 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:03.738483 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:03.738726 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:03.738729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:03.738731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:03.739309 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:03.739523 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:03.739742 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 18:30:03.740074 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":116,"threads_started":1}
I20260806 18:30:03.741521 15731 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:30:03.742585 15738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ae918f8f30249289e62d6a23115299a. Latest consensus state: current_term: 1 leader_uuid: "7ae918f8f30249289e62d6a23115299a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ae918f8f30249289e62d6a23115299a" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40455 } } }
I20260806 18:30:03.742712 15738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:03.742959 15737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ae918f8f30249289e62d6a23115299a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ae918f8f30249289e62d6a23115299a" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40455 } } }
I20260806 18:30:03.743055 15737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:03.743129 15747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:30:03.746470 15747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:30:03.752087 15747 catalog_manager.cc:1347] Generated new cluster ID: 4924fe2fdd7a4d0681cce05ce05fea6a
I20260806 18:30:03.752167 15747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:30:03.766579 15747 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:30:03.767627 15747 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:30:03.780292 15747 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ae918f8f30249289e62d6a23115299a: Generated new TSK 0
I20260806 18:30:03.781031 15747 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
I20260806 18:30:03.818122 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:40455
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:39881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:03.934978 15761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:03.935266 15761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:03.935308 15761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:03.939317 15761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:03.939380 15761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:03.939438 15761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:30:03.944859 15761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:39881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:40455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:03.945950 15761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:03.947007 15761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:30:03.956815 15761 server_base.cc:1034] running on GCE node
W20260806 18:30:03.956776 15770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:03.956779 15767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:03.956786 15768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:03.957397 15761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:03.958062 15761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:03.959218 15761 hybrid_clock.cc:648] HybridClock initialized: now 1786041003959194 us; error 36 us; skew 500 ppm
Aug 06 18:30:03 dist-test-slave-5bn2 krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041003, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:03.962060 15761 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:30:03.962978 15761 webserver.cc:460] Webserver started at http://127.7.67.193:36943/ using document root <none> and password file <none>
I20260806 18:30:03.963531 15761 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:03.963596 15761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:03.963801 15761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:03.965934 15761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "2e3a7d7b63424ccd9f48cf04f8bbd6e4"
format_stamp: "Formatted at 2026-08-06 18:30:03 on dist-test-slave-5bn2"
server_key: "3eef0a84751c604a201263a4786bfb8d"
server_key_iv: "af1b8017b50f7f36fb65e37b62b8a1c9"
server_key_version: "encryptionkey@0"
I20260806 18:30:03.966485 15761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "2e3a7d7b63424ccd9f48cf04f8bbd6e4"
format_stamp: "Formatted at 2026-08-06 18:30:03 on dist-test-slave-5bn2"
server_key: "3eef0a84751c604a201263a4786bfb8d"
server_key_iv: "af1b8017b50f7f36fb65e37b62b8a1c9"
server_key_version: "encryptionkey@0"
I20260806 18:30:03.970245 15761 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 18:30:03.972688 15783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:03.973862 15761 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:30:03.973987 15761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "2e3a7d7b63424ccd9f48cf04f8bbd6e4"
format_stamp: "Formatted at 2026-08-06 18:30:03 on dist-test-slave-5bn2"
server_key: "3eef0a84751c604a201263a4786bfb8d"
server_key_iv: "af1b8017b50f7f36fb65e37b62b8a1c9"
server_key_version: "encryptionkey@0"
I20260806 18:30:03.974088 15761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:03.992396 15761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:03.995730 15761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:03.995934 15761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:03.996560 15761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:03.997531 15761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:03.997591 15761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:03.997648 15761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:03.997674 15761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:04.006227 15761 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:43995
I20260806 18:30:04.006253 15938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:43995 every 8 connection(s)
I20260806 18:30:04.007192 15761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:30:04.015193 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 15761
I20260806 18:30:04.015321 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:30:04.015640 7439 external_mini_cluster.cc:1428] Setting key 14c520ae5f364a600a38498e5241d1a7
Aug 06 18:30:04 dist-test-slave-5bn2 krb5kdc[15612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041003, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:30:04.021418 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:04.009028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:48253 (local address 127.7.67.254:40455)
0806 18:30:04.009154 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:04.009158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:04.010002 (+ 844us) server_negotiation.cc:407] Connection header received
0806 18:30:04.011604 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:04.011608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:04.011673 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:04.011747 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:04.012990 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:04.013518 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:04.014652 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:04.014934 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:04.015036 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:04.017788 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:04.017792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:04.017794 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:04.017822 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:04.019283 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:04.019790 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:04.019794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:04.019796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:04.019883 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:04.020222 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:04.020225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:04.020227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:04.020421 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:04.020519 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:04.021164 (+ 645us) server_negotiation.cc:299] Negotiation successful
0806 18:30:04.021286 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:30:04.022578 15942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:04.009376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:40455 (local address 127.7.67.193:48253)
0806 18:30:04.009845 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:04.009890 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:04.011355 (+ 1465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:04.011939 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:04.011948 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:04.012320 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:04.012828 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:04.012842 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:04.013644 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:04.013646 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:04.014531 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:04.014540 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:04.015095 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:04.015099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:04.015272 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:04.015907 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:04.015924 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:04.017619 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:04.019398 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:04.019404 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:04.019422 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:04.019694 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:04.019976 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:04.019979 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:04.019982 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:04.020132 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:04.020532 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:04.020539 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:04.020873 (+ 334us) client_negotiation.cc:769] Sending connection context
0806 18:30:04.021098 (+ 225us) client_negotiation.cc:240] Negotiation successful
0806 18:30:04.021454 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":113,"threads_started":1}
I20260806 18:30:04.023983 15940 heartbeater.cc:347] Connected to a master server at 127.7.67.254:40455
I20260806 18:30:04.024271 15940 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:04.024914 15940 heartbeater.cc:511] Master 127.7.67.254:40455 requested a full tablet report, sending...
I20260806 18:30:04.026818 15675 ts_manager.cc:194] Registered new tserver with Master: 2e3a7d7b63424ccd9f48cf04f8bbd6e4 (127.7.67.193:43995)
WARNING: no policy specified for kudu/127.7.67.194@KRBTEST.COM; defaulting to no policy
I20260806 18:30:04.028172 15675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:48253
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
I20260806 18:30:04.044255 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:40455
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:39881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:04.158916 15947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:04.159179 15947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:04.159221 15947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:04.163257 15947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:04.163321 15947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:04.163381 15947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:30:04.168710 15947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:39881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:40455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:04.169806 15947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:04.170917 15947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:04.180701 15954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:04.180748 15953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:04.180732 15947 server_base.cc:1034] running on GCE node
W20260806 18:30:04.180708 15957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:04.181344 15947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:04.181947 15947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:04.183090 15947 hybrid_clock.cc:648] HybridClock initialized: now 1786041004183073 us; error 28 us; skew 500 ppm
Aug 06 18:30:04 dist-test-slave-5bn2 krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041004, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:04.185714 15947 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:30:04.186560 15947 webserver.cc:460] Webserver started at http://127.7.67.194:32941/ using document root <none> and password file <none>
I20260806 18:30:04.187144 15947 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:04.187211 15947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:04.187444 15947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:04.189746 15947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "a3f20810cd8b480094876ac50e82f3ea"
format_stamp: "Formatted at 2026-08-06 18:30:04 on dist-test-slave-5bn2"
server_key: "07f439ad9b86bed0d8e4555c653ac154"
server_key_iv: "b47cf742d65bb3910d4237a3b69cac88"
server_key_version: "encryptionkey@0"
I20260806 18:30:04.190327 15947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "a3f20810cd8b480094876ac50e82f3ea"
format_stamp: "Formatted at 2026-08-06 18:30:04 on dist-test-slave-5bn2"
server_key: "07f439ad9b86bed0d8e4555c653ac154"
server_key_iv: "b47cf742d65bb3910d4237a3b69cac88"
server_key_version: "encryptionkey@0"
I20260806 18:30:04.194000 15947 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:30:04.196399 15967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:04.197564 15947 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:30:04.197666 15947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "a3f20810cd8b480094876ac50e82f3ea"
format_stamp: "Formatted at 2026-08-06 18:30:04 on dist-test-slave-5bn2"
server_key: "07f439ad9b86bed0d8e4555c653ac154"
server_key_iv: "b47cf742d65bb3910d4237a3b69cac88"
server_key_version: "encryptionkey@0"
I20260806 18:30:04.197767 15947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:04.233276 15947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:04.236361 15947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:04.236552 15947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:04.237154 15947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:04.238006 15947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:04.238051 15947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:04.238094 15947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:04.238123 15947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:04.246737 15947 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:33905
I20260806 18:30:04.246769 16119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:33905 every 8 connection(s)
I20260806 18:30:04.247694 15947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:30:04.251789 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 15947
I20260806 18:30:04.251895 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:30:04.252189 7439 external_mini_cluster.cc:1428] Setting key 2dde1387b1ac94faf2ce7f764f10eb7e
Aug 06 18:30:04 dist-test-slave-5bn2 krb5kdc[15612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041004, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:30:04.261814 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:04.249458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:36183 (local address 127.7.67.254:40455)
0806 18:30:04.249620 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:04.249625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:04.250456 (+ 831us) server_negotiation.cc:407] Connection header received
0806 18:30:04.251815 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:04.251819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:04.251875 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:04.251968 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:04.253343 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:04.254068 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:04.255247 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:04.255557 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:04.255662 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:04.258312 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:04.258317 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:04.258320 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:04.258352 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:04.259886 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:04.260347 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:04.260351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:04.260353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:04.260457 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:04.260744 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:04.260747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:04.260748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:04.260972 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:04.261069 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:04.261515 (+ 446us) server_negotiation.cc:299] Negotiation successful
0806 18:30:04.261642 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:30:04.262542 16123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:04.249846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:40455 (local address 127.7.67.194:36183)
0806 18:30:04.250289 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:04.250330 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:04.251618 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:04.252142 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:04.252148 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:04.252530 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:04.253182 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:04.253202 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:04.254207 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:04.254209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:04.255137 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:04.255146 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:04.255710 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:04.255715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:04.255923 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:04.256519 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:04.256539 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:04.258136 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:04.259994 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:04.260000 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:04.260011 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:04.260258 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:04.260537 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:04.260539 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:04.260540 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:04.260666 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:04.261056 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:04.261061 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:04.261285 (+ 224us) client_negotiation.cc:769] Sending connection context
0806 18:30:04.261432 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 18:30:04.261677 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.7.67.195@KRBTEST.COM; defaulting to no policy
I20260806 18:30:04.263630 16121 heartbeater.cc:347] Connected to a master server at 127.7.67.254:40455
I20260806 18:30:04.263859 16121 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:04.264338 16121 heartbeater.cc:511] Master 127.7.67.254:40455 requested a full tablet report, sending...
I20260806 18:30:04.265368 15675 ts_manager.cc:194] Registered new tserver with Master: a3f20810cd8b480094876ac50e82f3ea (127.7.67.194:33905)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.195@KRBTEST.COM" created.
I20260806 18:30:04.265928 15675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:36183
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
I20260806 18:30:04.279968 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:40455
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:39881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:04.393895 16128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:04.394243 16128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:04.394290 16128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:04.398345 16128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:04.398412 16128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:04.398469 16128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:30:04.403745 16128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:39881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:40455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:04.404784 16128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:04.405781 16128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:04.415306 16135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:04.415313 16134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:04.415387 16138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:04.415776 16128 server_base.cc:1034] running on GCE node
I20260806 18:30:04.416198 16128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:04.416852 16128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:04.418001 16128 hybrid_clock.cc:648] HybridClock initialized: now 1786041004417977 us; error 39 us; skew 500 ppm
Aug 06 18:30:04 dist-test-slave-5bn2 krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041004, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:04.420753 16128 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:30:04.421698 16128 webserver.cc:460] Webserver started at http://127.7.67.195:38963/ using document root <none> and password file <none>
I20260806 18:30:04.422271 16128 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:04.422339 16128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:04.422542 16128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:04.424714 16128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "e18c27feb00f4ba2a0f89980a5759d95"
format_stamp: "Formatted at 2026-08-06 18:30:04 on dist-test-slave-5bn2"
server_key: "5d6ab0c1c4c0d6f5d750e95f5276b2c2"
server_key_iv: "cb3eda26b69d27dd21c6c211d1727872"
server_key_version: "encryptionkey@0"
I20260806 18:30:04.425275 16128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "e18c27feb00f4ba2a0f89980a5759d95"
format_stamp: "Formatted at 2026-08-06 18:30:04 on dist-test-slave-5bn2"
server_key: "5d6ab0c1c4c0d6f5d750e95f5276b2c2"
server_key_iv: "cb3eda26b69d27dd21c6c211d1727872"
server_key_version: "encryptionkey@0"
I20260806 18:30:04.429179 16128 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:30:04.431634 16147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:04.432822 16128 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:30:04.432957 16128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "e18c27feb00f4ba2a0f89980a5759d95"
format_stamp: "Formatted at 2026-08-06 18:30:04 on dist-test-slave-5bn2"
server_key: "5d6ab0c1c4c0d6f5d750e95f5276b2c2"
server_key_iv: "cb3eda26b69d27dd21c6c211d1727872"
server_key_version: "encryptionkey@0"
I20260806 18:30:04.433059 16128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:04.454499 16128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:04.457549 16128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:04.457729 16128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:04.458331 16128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:04.459199 16128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:04.459249 16128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:04.459302 16128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:04.459331 16128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:04.468034 16128 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:34841
I20260806 18:30:04.468065 16303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:34841 every 8 connection(s)
I20260806 18:30:04.468915 16128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:30:04.477219 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 16128
I20260806 18:30:04.477339 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:30:04.477613 7439 external_mini_cluster.cc:1428] Setting key 77409aebeeeafcdffd7ac375785c98e8
Aug 06 18:30:04 dist-test-slave-5bn2 krb5kdc[15612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041004, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:30:04.482396 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:04.470707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:47241 (local address 127.7.67.254:40455)
0806 18:30:04.470866 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:04.470872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:04.471706 (+ 834us) server_negotiation.cc:407] Connection header received
0806 18:30:04.473087 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:04.473091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:04.473150 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:04.473220 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:04.474366 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:04.474943 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:04.476105 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:04.476323 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:04.476412 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:04.479203 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:04.479213 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:04.479216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:04.479249 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:04.480621 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:04.481096 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:04.481098 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:04.481099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:04.481167 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:04.481465 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:04.481466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:04.481468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:04.481625 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:04.481694 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:04.482177 (+ 483us) server_negotiation.cc:299] Negotiation successful
0806 18:30:04.482272 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:30:04.483297 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:04.471081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:40455 (local address 127.7.67.195:47241)
0806 18:30:04.471558 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:04.471601 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:04.472881 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:04.473408 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:04.473416 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:04.473762 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:04.474226 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:04.474239 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:04.475070 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:04.475072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:04.475986 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:04.475996 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:04.476468 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:04.476473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:04.476627 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:04.477192 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:04.477211 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:04.479034 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:04.480729 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:04.480734 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:04.480746 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:04.481016 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:04.481232 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:04.481234 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:04.481235 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:04.481393 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:04.481738 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:04.481743 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:04.481976 (+ 233us) client_negotiation.cc:769] Sending connection context
0806 18:30:04.482135 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 18:30:04.482382 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":116,"threads_started":1}
I20260806 18:30:04.484408 16305 heartbeater.cc:347] Connected to a master server at 127.7.67.254:40455
I20260806 18:30:04.484622 16305 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:04.485071 16305 heartbeater.cc:511] Master 127.7.67.254:40455 requested a full tablet report, sending...
I20260806 18:30:04.486035 15675 ts_manager.cc:194] Registered new tserver with Master: e18c27feb00f4ba2a0f89980a5759d95 (127.7.67.195:34841)
I20260806 18:30:04.486619 15675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:47241
I20260806 18:30:04.491636 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/test-admin.keytab.
Aug 06 18:30:04 dist-test-slave-5bn2 krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:04.507328 7439 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 18:30:04.510439 16319 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:04.508816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54874 (local address 127.7.67.193:43995)
0806 18:30:04.509147 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:04.509164 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:04.509190 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:30:04.509248 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:04.509253 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:04.509469 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:04.509603 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:04.510172 (+ 569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54874: BlockingRecv error: recv got EOF from 127.0.0.1:54874 (error 108)
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":120,"threads_started":1}
W20260806 18:30:04.510484 7439 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/krb5cc' of type FILE
Aug 06 18:30:04 dist-test-slave-5bn2 krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:04.511624 7439 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 18:30:04 dist-test-slave-5bn2 krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:04.512473 7439 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 18:30:04 dist-test-slave-5bn2 krb5kdc[15612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.193@KRBTEST.COM
I20260806 18:30:04.521974 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:04.513404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54882 (local address 127.7.67.193:43995)
0806 18:30:04.513538 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:04.513542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:04.513639 (+ 97us) server_negotiation.cc:407] Connection header received
0806 18:30:04.513765 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:04.513767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:04.513808 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:04.513868 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:04.514395 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:04.515253 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:04.516095 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:04.516470 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:04.516565 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:04.518712 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:04.518722 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:04.518735 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:04.518759 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:04.520283 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:04.520695 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:04.520701 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:04.520702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:04.520792 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:04.521003 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:04.521005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:04.521006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:04.521254 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:04.521412 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:04.521644 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 18:30:04.521791 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 18:30:04.525818 16319 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:04.524901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54884 (local address 127.7.67.193:43995)
0806 18:30:04.525030 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:04.525034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:04.525103 (+ 69us) server_negotiation.cc:407] Connection header received
0806 18:30:04.525217 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:04.525220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:04.525260 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:04.525320 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:04.525726 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54884: BlockingRecv error: recv got EOF from 127.0.0.1:54884 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 18:30:04.526175 7439 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/krb5cc)
Aug 06 18:30:04 dist-test-slave-5bn2 krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:04.527210 7439 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 18:30:04 dist-test-slave-5bn2 krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:04.528085 7439 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 18:30:04 dist-test-slave-5bn2 krb5kdc[15612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.193@KRBTEST.COM
I20260806 18:30:04.536634 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:04.528928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54900 (local address 127.7.67.193:43995)
0806 18:30:04.529041 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:04.529046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:04.529120 (+ 74us) server_negotiation.cc:407] Connection header received
0806 18:30:04.529241 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:04.529244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:04.529297 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:04.529371 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:04.529917 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:04.530457 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:04.531372 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:04.531601 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:04.531690 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:04.533670 (+ 1980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:04.533673 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:04.533675 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:04.533699 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:04.535008 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:04.535439 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:04.535442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:04.535443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:04.535528 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:04.535771 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:04.535774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:04.535775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:04.535964 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:04.536098 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:04.536351 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 18:30:04.536456 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 18:30:04.539340 16319 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:04.538484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54908 (local address 127.7.67.193:43995)
0806 18:30:04.538606 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:04.538609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:04.538675 (+ 66us) server_negotiation.cc:407] Connection header received
0806 18:30:04.538760 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:04.538762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:04.538806 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:04.538906 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:04.539250 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54908: BlockingRecv error: recv got EOF from 127.0.0.1:54908 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 18:30:04.539460 7439 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786040973123132-7439-0/krb5kdc/krb5cc' of type FILE
Aug 06 18:30:04 dist-test-slave-5bn2 krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:04.540393 7439 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 18:30:04 dist-test-slave-5bn2 krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:04.541200 7439 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 18:30:04 dist-test-slave-5bn2 krb5kdc[15612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.193@KRBTEST.COM
I20260806 18:30:04.549167 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:04.541991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54916 (local address 127.7.67.193:43995)
0806 18:30:04.542114 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:04.542117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:04.542158 (+ 41us) server_negotiation.cc:407] Connection header received
0806 18:30:04.542255 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:04.542257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:04.542292 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:04.542431 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:04.542929 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:04.543433 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:04.544225 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:04.544438 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:04.544511 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:04.546477 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:04.546481 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:04.546483 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:04.546505 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:04.547790 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:04.548142 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:04.548144 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:04.548145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:04.548212 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:04.548485 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:04.548488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:04.548489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:04.548665 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:04.548771 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:04.548957 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 18:30:04.549048 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:30:04.550261 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 15761
I20260806 18:30:04.569837 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 15947
I20260806 18:30:04.584187 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 16128
I20260806 18:30:04.599272 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 15629
2026-08-06T18:30:04Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2325 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:30:04 dist-test-slave-5bn2 krb5kdc[16334](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:30:04 dist-test-slave-5bn2 krb5kdc[16334](info): set up 2 sockets
Aug 06 18:30:04 dist-test-slave-5bn2 krb5kdc[16334](info): commencing operation
krb5kdc: starting...
W20260806 18:30:05.713088 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:30:05 dist-test-slave-5bn2 krb5kdc[16334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:30:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:30:05Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.254.keytab.
Entry for principal oryx/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.254.keytab.
I20260806 18:30:05.836051 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:39723
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.254.keytab
--principal=oryx/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:36393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:39723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:05.950016 16350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:05.950331 16350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:05.950393 16350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:05.954392 16350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:30:05.954460 16350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:05.954486 16350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:05.954515 16350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:30:05.954540 16350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:30:05.959858 16350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:36393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:39723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.254.keytab
--principal=oryx/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:39723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:05.960901 16350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:05.961925 16350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:05.970292 16363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:05.970264 16358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:05.970299 16359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:05.970624 16350 server_base.cc:1034] running on GCE node
I20260806 18:30:05.971223 16350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:05.972339 16350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:05.973488 16350 hybrid_clock.cc:648] HybridClock initialized: now 1786041005973470 us; error 43 us; skew 500 ppm
Aug 06 18:30:05 dist-test-slave-5bn2 krb5kdc[16334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041005, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:05.976131 16350 init.cc:377] Logged in from keytab as oryx/127.7.67.254@KRBTEST.COM (short username oryx)
I20260806 18:30:05.976974 16350 webserver.cc:460] Webserver started at http://127.7.67.254:42185/ using document root <none> and password file <none>
I20260806 18:30:05.977504 16350 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:05.977568 16350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:05.977771 16350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:05.979799 16350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "c6a620695abe4ec7b4577e405a25f199"
format_stamp: "Formatted at 2026-08-06 18:30:05 on dist-test-slave-5bn2"
server_key: "70351cacfb7f00470fcaf7e1c7ea2e57"
server_key_iv: "b7f6ab9488cab884d2e56ea396af104d"
server_key_version: "encryptionkey@0"
I20260806 18:30:05.980334 16350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "c6a620695abe4ec7b4577e405a25f199"
format_stamp: "Formatted at 2026-08-06 18:30:05 on dist-test-slave-5bn2"
server_key: "70351cacfb7f00470fcaf7e1c7ea2e57"
server_key_iv: "b7f6ab9488cab884d2e56ea396af104d"
server_key_version: "encryptionkey@0"
I20260806 18:30:05.984062 16350 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 18:30:05.986364 16372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:05.987587 16350 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:30:05.987720 16350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "c6a620695abe4ec7b4577e405a25f199"
format_stamp: "Formatted at 2026-08-06 18:30:05 on dist-test-slave-5bn2"
server_key: "70351cacfb7f00470fcaf7e1c7ea2e57"
server_key_iv: "b7f6ab9488cab884d2e56ea396af104d"
server_key_version: "encryptionkey@0"
I20260806 18:30:05.987816 16350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:06.013186 16350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:06.016110 16350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:06.016301 16350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:06.023178 16350 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:39723
I20260806 18:30:06.023432 16440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:39723 every 8 connection(s)
I20260806 18:30:06.024178 16350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:30:06.027181 16441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:06.032831 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 16350
I20260806 18:30:06.032864 16441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199: Bootstrap starting.
I20260806 18:30:06.032963 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:30:06.033216 7439 external_mini_cluster.cc:1428] Setting key 5a1f3686d1552a6d25e0ddcbedc0047d
I20260806 18:30:06.035643 16441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:06.036576 16441 log.cc:824] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:06.038935 16441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199: No bootstrap required, opened a new log
Aug 06 18:30:06 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM
I20260806 18:30:06.043247 16441 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } }
I20260806 18:30:06.043509 16441 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:06.043581 16441 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6a620695abe4ec7b4577e405a25f199, State: Initialized, Role: FOLLOWER
I20260806 18:30:06.044080 16441 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } }
I20260806 18:30:06.044260 16441 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:30:06.044332 16441 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:30:06.044435 16441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:06.045364 16441 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } }
I20260806 18:30:06.045759 16441 leader_election.cc:302] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6a620695abe4ec7b4577e405a25f199; no voters:
I20260806 18:30:06.046172 16441 leader_election.cc:288] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:30:06.046254 16447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:06.046500 16447 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [term 1 LEADER]: Becoming Leader. State: Replica: c6a620695abe4ec7b4577e405a25f199, State: Running, Role: LEADER
I20260806 18:30:06.046888 16447 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } }
I20260806 18:30:06.047638 16441 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:30:06.048615 16449 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6a620695abe4ec7b4577e405a25f199" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } }
I20260806 18:30:06.048738 16449 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:06.048902 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:06.034651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49572 (local address 127.7.67.254:39723)
0806 18:30:06.035127 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:06.035147 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:06.035191 (+ 44us) server_negotiation.cc:407] Connection header received
0806 18:30:06.036075 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:06.036098 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:06.036535 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:06.036914 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:06.037729 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:06.039200 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:06.040059 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:06.040770 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:06.040934 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:06.043538 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:06.043544 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:06.043562 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:06.043600 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:06.045334 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:06.045819 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:06.045824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:06.045829 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:06.045972 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:06.046337 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:06.046340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:06.046342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:06.046899 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:06.047106 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:06.047377 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 18:30:06.047860 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":163,"threads_started":1}
I20260806 18:30:06.049264 16452 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6a620695abe4ec7b4577e405a25f199. Latest consensus state: current_term: 1 leader_uuid: "c6a620695abe4ec7b4577e405a25f199" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } }
I20260806 18:30:06.049412 16452 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:06.050033 16460 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:30:06.053342 16460 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:30:06.058974 16460 catalog_manager.cc:1347] Generated new cluster ID: 046fb588ad9f453898a7561cc299683e
I20260806 18:30:06.059058 16460 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:30:06.064036 16460 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:30:06.065068 16460 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:30:06.069490 16460 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199: Generated new TSK 0
I20260806 18:30:06.070279 16460 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.193.keytab.
Entry for principal oryx/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.193.keytab.
I20260806 18:30:06.103888 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39723
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.193.keytab
--principal=oryx/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:36393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:06.235945 16478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:06.236228 16478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:06.236289 16478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:06.240331 16478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:06.240398 16478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:06.240480 16478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:30:06.245878 16478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:36393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.193.keytab
--principal=oryx/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:06.247051 16478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:06.248205 16478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:06.258051 16489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:06.258051 16486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:06.258044 16487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:06.258102 16478 server_base.cc:1034] running on GCE node
I20260806 18:30:06.258797 16478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:06.259470 16478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:06.260636 16478 hybrid_clock.cc:648] HybridClock initialized: now 1786041006260617 us; error 34 us; skew 500 ppm
Aug 06 18:30:06 dist-test-slave-5bn2 krb5kdc[16334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041006, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:06.263319 16478 init.cc:377] Logged in from keytab as oryx/127.7.67.193@KRBTEST.COM (short username oryx)
I20260806 18:30:06.264221 16478 webserver.cc:460] Webserver started at http://127.7.67.193:37127/ using document root <none> and password file <none>
I20260806 18:30:06.264771 16478 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:06.264838 16478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:06.265053 16478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:06.267159 16478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "04c0f654471b4bf19e477ac1f30b535b"
format_stamp: "Formatted at 2026-08-06 18:30:06 on dist-test-slave-5bn2"
server_key: "a7b0867163bdfe558058dfd73d69bf12"
server_key_iv: "bea5ca5911df1b8655ef291f1ee767ed"
server_key_version: "encryptionkey@0"
I20260806 18:30:06.267724 16478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "04c0f654471b4bf19e477ac1f30b535b"
format_stamp: "Formatted at 2026-08-06 18:30:06 on dist-test-slave-5bn2"
server_key: "a7b0867163bdfe558058dfd73d69bf12"
server_key_iv: "bea5ca5911df1b8655ef291f1ee767ed"
server_key_version: "encryptionkey@0"
I20260806 18:30:06.271411 16478 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260806 18:30:06.273758 16500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:06.274981 16478 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:30:06.275102 16478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "04c0f654471b4bf19e477ac1f30b535b"
format_stamp: "Formatted at 2026-08-06 18:30:06 on dist-test-slave-5bn2"
server_key: "a7b0867163bdfe558058dfd73d69bf12"
server_key_iv: "bea5ca5911df1b8655ef291f1ee767ed"
server_key_version: "encryptionkey@0"
I20260806 18:30:06.275202 16478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:06.294955 16478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:06.297886 16478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:06.298085 16478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:06.298691 16478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:06.299561 16478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:06.299609 16478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:06.299664 16478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:06.299691 16478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:06.308287 16478 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:45973
I20260806 18:30:06.308315 16655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:45973 every 8 connection(s)
I20260806 18:30:06.309429 16478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:30:06.311401 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 16478
I20260806 18:30:06.311546 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:30:06.311869 7439 external_mini_cluster.cc:1428] Setting key 8d9aac5b4997d47faa72f5fd17439538
Aug 06 18:30:06 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041006, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.7.67.194@KRBTEST.COM; defaulting to no policy
I20260806 18:30:06.324191 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:06.311310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:47245 (local address 127.7.67.254:39723)
0806 18:30:06.311440 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:06.311443 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:06.312583 (+ 1140us) server_negotiation.cc:407] Connection header received
0806 18:30:06.314164 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:06.314168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:06.314220 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:06.314294 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:06.315614 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:06.316148 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:06.317426 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:06.317651 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:06.317722 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:06.320524 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:06.320528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:06.320531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:06.320562 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:06.322065 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:06.322584 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:06.322587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:06.322589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:06.322681 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:06.323023 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:06.323027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:06.323028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:06.323275 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:06.323382 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:06.323902 (+ 520us) server_negotiation.cc:299] Negotiation successful
0806 18:30:06.324051 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:30:06.325093 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:06.311818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.193:47245)
0806 18:30:06.312402 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:06.312480 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:06.313952 (+ 1472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:06.314476 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:06.314484 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:06.314873 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:06.315440 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:06.315471 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:06.316273 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:06.316277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:06.317329 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:06.317340 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:06.317799 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:06.317803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:06.318018 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:06.318616 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:06.318636 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:06.320339 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:06.322192 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:06.322198 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:06.322211 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:06.322474 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:06.322768 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:06.322770 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:06.322772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:06.322939 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:06.323369 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:06.323374 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:06.323649 (+ 275us) client_negotiation.cc:769] Sending connection context
0806 18:30:06.323821 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 18:30:06.324112 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":178,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.194@KRBTEST.COM" created.
I20260806 18:30:06.326402 16656 heartbeater.cc:347] Connected to a master server at 127.7.67.254:39723
I20260806 18:30:06.326656 16656 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:06.327291 16656 heartbeater.cc:511] Master 127.7.67.254:39723 requested a full tablet report, sending...
I20260806 18:30:06.329080 16396 ts_manager.cc:194] Registered new tserver with Master: 04c0f654471b4bf19e477ac1f30b535b (127.7.67.193:45973)
I20260806 18:30:06.330312 16396 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:47245
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.194.keytab.
Entry for principal oryx/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.194.keytab.
I20260806 18:30:06.342043 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39723
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.194.keytab
--principal=oryx/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:36393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:06.455489 16666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:06.455721 16666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:06.455761 16666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:06.459672 16666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:06.459726 16666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:06.459782 16666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:30:06.464886 16666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:36393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.194.keytab
--principal=oryx/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:06.465854 16666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:06.466812 16666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:06.476199 16677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:06.476192 16675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:06.476192 16672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:06.476572 16666 server_base.cc:1034] running on GCE node
I20260806 18:30:06.476946 16666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:06.477537 16666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:06.478729 16666 hybrid_clock.cc:648] HybridClock initialized: now 1786041006478705 us; error 35 us; skew 500 ppm
Aug 06 18:30:06 dist-test-slave-5bn2 krb5kdc[16334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041006, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:06.481518 16666 init.cc:377] Logged in from keytab as oryx/127.7.67.194@KRBTEST.COM (short username oryx)
I20260806 18:30:06.482429 16666 webserver.cc:460] Webserver started at http://127.7.67.194:45123/ using document root <none> and password file <none>
I20260806 18:30:06.483018 16666 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:06.483088 16666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:06.483290 16666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:06.485317 16666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "9a703950a82f49cb8d102abf747e0f44"
format_stamp: "Formatted at 2026-08-06 18:30:06 on dist-test-slave-5bn2"
server_key: "09fd2f616ecde7e1199de99590f20894"
server_key_iv: "249913cacda9aebd0f57064283933fed"
server_key_version: "encryptionkey@0"
I20260806 18:30:06.485867 16666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "9a703950a82f49cb8d102abf747e0f44"
format_stamp: "Formatted at 2026-08-06 18:30:06 on dist-test-slave-5bn2"
server_key: "09fd2f616ecde7e1199de99590f20894"
server_key_iv: "249913cacda9aebd0f57064283933fed"
server_key_version: "encryptionkey@0"
I20260806 18:30:06.489609 16666 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:30:06.492094 16688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:06.493368 16666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:30:06.493495 16666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "9a703950a82f49cb8d102abf747e0f44"
format_stamp: "Formatted at 2026-08-06 18:30:06 on dist-test-slave-5bn2"
server_key: "09fd2f616ecde7e1199de99590f20894"
server_key_iv: "249913cacda9aebd0f57064283933fed"
server_key_version: "encryptionkey@0"
I20260806 18:30:06.493597 16666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:06.524780 16666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:06.527863 16666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:06.528059 16666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:06.528746 16666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:06.529635 16666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:06.529685 16666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:06.529740 16666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:06.529768 16666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:06.538432 16666 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:32907
I20260806 18:30:06.538457 16838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:32907 every 8 connection(s)
I20260806 18:30:06.539427 16666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
Aug 06 18:30:06 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041006, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM
I20260806 18:30:06.549420 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 16666
I20260806 18:30:06.549527 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:30:06.549852 7439 external_mini_cluster.cc:1428] Setting key 23d7054b44e7cdcb33b7c3bfbad822be
I20260806 18:30:06.553447 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:06.541247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:43615 (local address 127.7.67.254:39723)
0806 18:30:06.541423 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:06.541428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:06.542451 (+ 1023us) server_negotiation.cc:407] Connection header received
0806 18:30:06.543734 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:06.543737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:06.543788 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:06.543868 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:06.545053 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:06.545593 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:06.546730 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:06.547002 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:06.547081 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:06.549737 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:06.549741 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:06.549744 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:06.549775 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:06.551504 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:06.551979 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:06.551982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:06.551984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:06.552073 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:06.552365 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:06.552368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:06.552369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:06.552594 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:06.552690 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:06.553147 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 18:30:06.553270 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:30:06.554425 16844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:06.541785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.194:43615)
0806 18:30:06.542266 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:06.542312 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:06.543506 (+ 1194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:06.544027 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:06.544033 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:06.544400 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:06.544904 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:06.544918 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:06.545696 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:06.545699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:06.546623 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:06.546633 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:06.547170 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:06.547175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:06.547328 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:06.547934 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:06.547950 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:06.549532 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:06.551612 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:06.551617 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:06.551630 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:06.551883 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:06.552156 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:06.552159 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:06.552160 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:06.552290 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:06.552677 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:06.552682 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:06.552925 (+ 243us) client_negotiation.cc:769] Sending connection context
0806 18:30:06.553081 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 18:30:06.553342 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":129,"threads_started":1}
I20260806 18:30:06.555606 16839 heartbeater.cc:347] Connected to a master server at 127.7.67.254:39723
I20260806 18:30:06.555871 16839 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:06.556445 16839 heartbeater.cc:511] Master 127.7.67.254:39723 requested a full tablet report, sending...
I20260806 18:30:06.557632 16396 ts_manager.cc:194] Registered new tserver with Master: 9a703950a82f49cb8d102abf747e0f44 (127.7.67.194:32907)
I20260806 18:30:06.558254 16396 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:43615
WARNING: no policy specified for oryx/127.7.67.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.195.keytab.
Entry for principal oryx/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.195.keytab.
I20260806 18:30:06.577689 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39723
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.195.keytab
--principal=oryx/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:36393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:06.692546 16847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:06.692812 16847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:06.692870 16847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:06.696835 16847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:06.696898 16847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:06.696973 16847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:30:06.702149 16847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:36393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.195.keytab
--principal=oryx/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:39723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:06.703245 16847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:06.704281 16847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:06.714025 16856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:06.713964 16853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:06.713966 16858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:06.714114 16847 server_base.cc:1034] running on GCE node
I20260806 18:30:06.714700 16847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:06.715370 16847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:06.716533 16847 hybrid_clock.cc:648] HybridClock initialized: now 1786041006716516 us; error 34 us; skew 500 ppm
Aug 06 18:30:06 dist-test-slave-5bn2 krb5kdc[16334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041006, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:06.719244 16847 init.cc:377] Logged in from keytab as oryx/127.7.67.195@KRBTEST.COM (short username oryx)
I20260806 18:30:06.720196 16847 webserver.cc:460] Webserver started at http://127.7.67.195:36673/ using document root <none> and password file <none>
I20260806 18:30:06.720749 16847 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:06.720817 16847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:06.721045 16847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:06.723161 16847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "09fb11cf02c9434fb56d3d390a1a3536"
format_stamp: "Formatted at 2026-08-06 18:30:06 on dist-test-slave-5bn2"
server_key: "d61a334f1aca2d66524ef57bb0475507"
server_key_iv: "1ded5a65224ea11ce6ef5a7b6d394877"
server_key_version: "encryptionkey@0"
I20260806 18:30:06.723712 16847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "09fb11cf02c9434fb56d3d390a1a3536"
format_stamp: "Formatted at 2026-08-06 18:30:06 on dist-test-slave-5bn2"
server_key: "d61a334f1aca2d66524ef57bb0475507"
server_key_iv: "1ded5a65224ea11ce6ef5a7b6d394877"
server_key_version: "encryptionkey@0"
I20260806 18:30:06.727881 16847 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:30:06.730334 16866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:06.731609 16847 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:30:06.731734 16847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "09fb11cf02c9434fb56d3d390a1a3536"
format_stamp: "Formatted at 2026-08-06 18:30:06 on dist-test-slave-5bn2"
server_key: "d61a334f1aca2d66524ef57bb0475507"
server_key_iv: "1ded5a65224ea11ce6ef5a7b6d394877"
server_key_version: "encryptionkey@0"
I20260806 18:30:06.731830 16847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:06.747510 16847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:06.750449 16847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:06.750633 16847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:06.751267 16847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:06.752106 16847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:06.752156 16847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:06.752211 16847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:06.752235 16847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:06.760886 16847 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:45549
I20260806 18:30:06.760910 17020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:45549 every 8 connection(s)
I20260806 18:30:06.761773 16847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:30:06.764770 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 16847
I20260806 18:30:06.764882 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:30:06.765165 7439 external_mini_cluster.cc:1428] Setting key fc30196530e0074c7864df519a6d7f2d
Aug 06 18:30:06 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041006, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM
I20260806 18:30:06.775213 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:06.763573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:33723 (local address 127.7.67.254:39723)
0806 18:30:06.763713 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:06.763716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:06.764514 (+ 798us) server_negotiation.cc:407] Connection header received
0806 18:30:06.765850 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:06.765854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:06.765913 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:06.765986 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:06.767179 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:06.767731 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:06.768846 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:06.769069 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:06.769156 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:06.772017 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:06.772022 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:06.772025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:06.772053 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:06.773432 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:06.773877 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:06.773879 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:06.773881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:06.773950 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:06.774221 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:06.774223 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:06.774224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:06.774388 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:06.774458 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:06.774930 (+ 472us) server_negotiation.cc:299] Negotiation successful
0806 18:30:06.775057 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:30:06.776059 17023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:06.763928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.195:33723)
0806 18:30:06.764366 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:06.764410 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:06.765643 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:06.766172 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:06.766179 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:06.766531 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:06.767047 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:06.767063 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:06.767851 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:06.767854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:06.768730 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:06.768739 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:06.769207 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:06.769211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:06.769375 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:06.769933 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:06.769953 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:06.771828 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:06.773523 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:06.773528 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:06.773540 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:06.773799 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:06.774013 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:06.774015 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:06.774016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:06.774157 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:06.774459 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:06.774464 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:06.774702 (+ 238us) client_negotiation.cc:769] Sending connection context
0806 18:30:06.774883 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 18:30:06.775240 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":104,"threads_started":1}
I20260806 18:30:06.777122 17021 heartbeater.cc:347] Connected to a master server at 127.7.67.254:39723
I20260806 18:30:06.777376 17021 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:06.777869 17021 heartbeater.cc:511] Master 127.7.67.254:39723 requested a full tablet report, sending...
I20260806 18:30:06.778961 16396 ts_manager.cc:194] Registered new tserver with Master: 09fb11cf02c9434fb56d3d390a1a3536 (127.7.67.195:45549)
I20260806 18:30:06.779511 16396 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:33723
I20260806 18:30:06.790484 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.67.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.253.keytab.
Entry for principal oryx/127.7.67.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.253.keytab.
I20260806 18:30:06.817021 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.253:38821
--webserver_interface=127.7.67.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.253.keytab
--principal=oryx/127.7.67.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:36393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:39723,127.7.67.253:38821
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:06.930195 17027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:06.930456 17027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:06.930496 17027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:06.934453 17027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:30:06.934510 17027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:06.934528 17027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:06.934543 17027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:30:06.934563 17027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:30:06.939801 17027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:36393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:39723,127.7.67.253:38821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.253.keytab
--principal=oryx/127.7.67.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.253:38821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.67.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
W20260806 18:30:06.940001 17027 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.67.254:39723,127.7.67.253:38821'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 18:30:06.940822 17027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:06.941795 17027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:06.950366 17038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:06.950371 17035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:06.950371 17036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:06.950601 17027 server_base.cc:1034] running on GCE node
I20260806 18:30:06.951294 17027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:06.952373 17027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:06.953548 17027 hybrid_clock.cc:648] HybridClock initialized: now 1786041006953532 us; error 33 us; skew 500 ppm
Aug 06 18:30:06 dist-test-slave-5bn2 krb5kdc[16334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041006, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:06.956281 17027 init.cc:377] Logged in from keytab as oryx/127.7.67.253@KRBTEST.COM (short username oryx)
I20260806 18:30:06.957172 17027 webserver.cc:460] Webserver started at http://127.7.67.253:46293/ using document root <none> and password file <none>
I20260806 18:30:06.957720 17027 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:06.957783 17027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:06.957988 17027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:06.960109 17027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/data/instance:
uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11"
format_stamp: "Formatted at 2026-08-06 18:30:06 on dist-test-slave-5bn2"
server_key: "5bf2a32e4a323b0c02d89bfb4fc6fde1"
server_key_iv: "4d5394a6f2506141e24b57443579e552"
server_key_version: "encryptionkey@0"
I20260806 18:30:06.960666 17027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/wal/instance:
uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11"
format_stamp: "Formatted at 2026-08-06 18:30:06 on dist-test-slave-5bn2"
server_key: "5bf2a32e4a323b0c02d89bfb4fc6fde1"
server_key_iv: "4d5394a6f2506141e24b57443579e552"
server_key_version: "encryptionkey@0"
I20260806 18:30:06.964382 17027 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:30:06.966777 17046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:06.968010 17027 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:30:06.968122 17027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/wal
uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11"
format_stamp: "Formatted at 2026-08-06 18:30:06 on dist-test-slave-5bn2"
server_key: "5bf2a32e4a323b0c02d89bfb4fc6fde1"
server_key_iv: "4d5394a6f2506141e24b57443579e552"
server_key_version: "encryptionkey@0"
I20260806 18:30:06.968222 17027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:06.985503 17027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:06.988561 17027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:06.988744 17027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:06.997277 17027 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.253:38821
I20260806 18:30:06.997570 17121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.253:38821 every 8 connection(s)
I20260806 18:30:06.998399 17027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/data/info.pb
I20260806 18:30:07.001428 17123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:07.003751 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 17027
I20260806 18:30:07.005834 17123 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041006, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.253@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.67.253@KRBTEST.COM
I20260806 18:30:07.017984 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.007638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54030 (local address 127.7.67.254:39723)
0806 18:30:07.007768 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.007772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.008296 (+ 524us) server_negotiation.cc:407] Connection header received
0806 18:30:07.008436 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.008438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.008490 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.008555 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.009402 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.010020 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.011296 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.011554 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.011666 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.014105 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.014110 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.014113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.014146 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.015767 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.016401 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.016405 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.016406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.016495 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.016835 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.016839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.016841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.017085 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.017186 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.017650 (+ 464us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.017786 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:30:07.018946 17128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.005405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41550 (local address 127.7.67.253:38821)
0806 18:30:07.005999 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.006016 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.006056 (+ 40us) server_negotiation.cc:407] Connection header received
0806 18:30:07.006752 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.006771 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.007174 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.007534 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.008390 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.009725 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.010677 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.011400 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.011509 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.014562 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.014568 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.014581 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.014606 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.016281 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.016694 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.016701 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.016702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.016778 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.017035 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.017039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.017040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.017404 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.017600 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.017818 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.018006 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":444,"thread_start_us":270,"threads_started":1}
I20260806 18:30:07.018947 17129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.007581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54030)
0806 18:30:07.008132 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.008189 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.008323 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.008619 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.008625 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.009060 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.009257 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.009270 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.010173 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.010176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.011183 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.011193 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.011704 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.011706 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.011778 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.012366 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.012382 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.013921 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.015888 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.015899 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.015911 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.016286 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.016583 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.016586 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.016589 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.016736 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.017198 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.017206 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.017418 (+ 212us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.017584 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.017947 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":38,"thread_start_us":260,"threads_started":1}
I20260806 18:30:07.020416 17123 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041006, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.253@KRBTEST.COM for oryx/127.7.67.253@KRBTEST.COM
I20260806 18:30:07.028836 17129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.020954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:38821 (local address 127.0.0.1:41564)
0806 18:30:07.021059 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.021072 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.021148 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.021441 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.021443 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.021574 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.021670 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.021675 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.022340 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.022343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.022915 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.022926 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.024166 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.024169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.024243 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.024675 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.024688 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.025881 (+ 1193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.027512 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.027515 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.027516 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.027691 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.027933 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.027935 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.027937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.027998 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.028395 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.028398 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.028487 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.028587 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.028706 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 18:30:07.028947 17128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.021209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41564 (local address 127.7.67.253:38821)
0806 18:30:07.021292 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.021294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.021302 (+ 8us) server_negotiation.cc:407] Connection header received
0806 18:30:07.021323 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.021325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.021354 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.021402 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.021751 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.022246 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.023014 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.024066 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.024168 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.026002 (+ 1834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.026006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.026009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.026035 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.027419 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.027770 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.027773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.027775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.027858 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.028063 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.028066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.028067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.028272 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.028369 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.028639 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.028774 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":29}
I20260806 18:30:07.032573 17123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: Bootstrap starting.
I20260806 18:30:07.035386 17123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:07.036247 17123 log.cc:824] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:07.038185 17123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: No bootstrap required, opened a new log
I20260806 18:30:07.041325 17123 raft_consensus.cc:348] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } }
I20260806 18:30:07.041548 17123 raft_consensus.cc:374] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:07.041582 17123 raft_consensus.cc:729] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bec8fce3919c48a4b5d7e5cfb2a96b11, State: Initialized, Role: FOLLOWER
I20260806 18:30:07.041989 17123 consensus_queue.cc:260] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } }
I20260806 18:30:07.042500 17131 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } } }
I20260806 18:30:07.042623 17131 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:30:07.043278 17123 sys_catalog.cc:564] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 18:30:07.048372 17145 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:30:07.048565 17145 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:30:07.050312 17027 master_runner.cc:428] Detected that this master bec8fce3919c48a4b5d7e5cfb2a96b11 is joining an existing cluster
I20260806 18:30:07.050383 17027 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.67.253:38821
I20260806 18:30:07.060357 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.052886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54032 (local address 127.7.67.254:39723)
0806 18:30:07.053038 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.053043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.053214 (+ 171us) server_negotiation.cc:407] Connection header received
0806 18:30:07.053379 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.053382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.053438 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.053517 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.054064 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.054611 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.055620 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.055835 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.055922 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.057088 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.057091 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.057094 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.057122 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.058525 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.059019 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.059030 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.059032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.059130 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.059397 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.059400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.059402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.059583 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.059669 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.060053 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.060178 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:30:07.061146 17151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.052806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54032)
0806 18:30:07.053116 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.053130 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.053243 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.053531 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.053534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.053768 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.053919 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.053927 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.054716 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.054719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.055494 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.055505 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.055933 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.055936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.056033 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.056594 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.056606 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.056949 (+ 343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.058627 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.058630 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.058633 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.058904 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.059218 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.059222 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.059224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.059312 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.059672 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.059675 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.059790 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.060804 (+ 1014us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.060950 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":35,"thread_start_us":103,"threads_started":1}
I20260806 18:30:07.062011 17152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.052806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:38821 (local address 127.0.0.1:41574)
0806 18:30:07.053355 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.053367 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.053473 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.053747 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.053748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.053928 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.054031 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.054039 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.055002 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.055005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.056184 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.056196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.056832 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.056835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.056923 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.057843 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.057858 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.058283 (+ 425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.060466 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.060470 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.060472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.060745 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.061010 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.061020 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.061022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.061127 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.061516 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.061518 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.061612 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.061713 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.061837 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":330,"thread_start_us":93,"threads_started":1}
I20260806 18:30:07.062546 17128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.052902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41574 (local address 127.7.67.253:38821)
0806 18:30:07.053123 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.053128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.053436 (+ 308us) server_negotiation.cc:407] Connection header received
0806 18:30:07.053580 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.053583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.053619 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.053658 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.054123 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.054889 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.056307 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.056630 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.056700 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.058429 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.058433 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.058436 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.058462 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.060356 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.060840 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.060843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.060845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.060926 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.061220 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.061224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.061226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.061422 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.061522 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.062362 (+ 840us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.062453 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041005, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.254@KRBTEST.COM for oryx/127.7.67.253@KRBTEST.COM
I20260806 18:30:07.074599 17128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.065844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41588 (local address 127.7.67.253:38821)
0806 18:30:07.065960 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.065963 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.066321 (+ 358us) server_negotiation.cc:407] Connection header received
0806 18:30:07.066501 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.066504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.066559 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.066622 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.067296 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.067776 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.068610 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.068823 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.068905 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.071272 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.071276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.071278 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.071302 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.072672 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.073218 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.073222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.073224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.073317 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.073604 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.073607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.073609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.073822 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.073919 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.074322 (+ 403us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.074437 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 18:30:07.074668 17153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.065828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:38821 (local address 127.0.0.1:41588)
0806 18:30:07.066206 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.066248 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.066376 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.066669 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.066675 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.067080 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.067190 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.067196 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.067872 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.067876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.068502 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.068510 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.068917 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.068921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.069039 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.069521 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.069533 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.071135 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.072778 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.072789 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.072795 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.073103 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.073414 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.073417 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.073419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.073521 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.073936 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.073944 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.074102 (+ 158us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.074269 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.074487 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":110,"threads_started":1}
I20260806 18:30:07.076455 16396 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.67.253:38821: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11"
member_type: NON_VOTER
last_known_addr {
host: "127.7.67.253"
port: 38821
}
attrs {
promote: true
}
}
dest_uuid: "c6a620695abe4ec7b4577e405a25f199"
cas_config_opid_index: -1
I20260806 18:30:07.077145 16396 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: NON_VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: true } }
I20260806 18:30:07.078904 17156 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bec8fce3919c48a4b5d7e5cfb2a96b11 (127.7.67.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: NON_VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: true } } }
I20260806 18:30:07.079610 17086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:07.080735 17156 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.67.253:38821
I20260806 18:30:07.080783 17086 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 1 FOLLOWER]: Refusing update from remote peer c6a620695abe4ec7b4577e405a25f199: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 18:30:07.081094 17156 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c6a620695abe4ec7b4577e405a25f199" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: NON_VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: true } } }
I20260806 18:30:07.081197 17156 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:07.081393 17027 master.cc:537] Master@127.7.67.253:38821 shutting down...
I20260806 18:30:07.081727 17156 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: NON_VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:30:07.083825 16374 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 18:30:07.084738 16374 consensus_peers.cc:487] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 -> Peer bec8fce3919c48a4b5d7e5cfb2a96b11 (127.7.67.253:38821): Couldn't send request to peer bec8fce3919c48a4b5d7e5cfb2a96b11. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 18:30:07.085647 17027 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:30:07.085923 17027 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:07.086052 17027 tablet_replica.cc:330] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: stopping tablet replica
WARNING: no policy specified for oryx/127.7.67.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.252@KRBTEST.COM" created.
I20260806 18:30:07.100401 17027 master.cc:552] Master@127.7.67.253:38821 shutdown complete.
I20260806 18:30:07.100458 17027 master_runner.cc:305] Clearing existing system tablet
I20260806 18:30:07.100700 17027 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:07.100750 17027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:07.101824 17027 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:30:07.102579 17166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:07.102799 17027 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:07.102977 17027 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:30:07.103035 17027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/wal
uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11"
format_stamp: "Formatted at 2026-08-06 18:30:06 on dist-test-slave-5bn2"
server_key: "5bf2a32e4a323b0c02d89bfb4fc6fde1"
server_key_iv: "4d5394a6f2506141e24b57443579e552"
server_key_version: "encryptionkey@0"
I20260806 18:30:07.104976 17027 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:07.107826 17027 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 18:30:07.107926 17027 log.cc:1196] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 18:30:07.108196 17027 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: Deleting consensus metadata
I20260806 18:30:07.108364 17027 master_runner.cc:315] Copying system tablet from 127.7.67.254:39723
I20260806 18:30:07.109756 17027 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: tablet copy: Beginning tablet copy session from remote peer at address 127.7.67.254:39723
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.252.keytab.
Entry for principal oryx/127.7.67.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.252.keytab.
I20260806 18:30:07.115482 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.252:40865
--webserver_interface=127.7.67.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.252.keytab
--principal=oryx/127.7.67.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:36393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:39723,127.7.67.253:38821,127.7.67.252:40865
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:30:07.118490 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.110517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54042 (local address 127.7.67.254:39723)
0806 18:30:07.110665 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.110669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.110944 (+ 275us) server_negotiation.cc:407] Connection header received
0806 18:30:07.111096 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.111098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.111148 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.111219 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.111707 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.112294 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.113463 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.113685 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.113768 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.115039 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.115043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.115046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.115075 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.116805 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.117330 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.117332 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.117334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.117416 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.117689 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.117692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.117694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.117874 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.117960 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.118218 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.118328 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:30:07.118501 17176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.110405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54042)
0806 18:30:07.110811 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.110825 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.110995 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.111251 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.111254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.111465 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.111603 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.111610 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.112404 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.112408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.113347 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.113359 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.113792 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.113796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.113907 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.114472 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.114488 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.114843 (+ 355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.116922 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.116926 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.116928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.117190 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.117512 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.117515 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.117517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.117606 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.117964 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.117967 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.118064 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.118149 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.118300 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":122,"threads_started":1}
I20260806 18:30:07.119341 16422 tablet_copy_service.cc:140] P c6a620695abe4ec7b4577e405a25f199: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bec8fce3919c48a4b5d7e5cfb2a96b11 ({username='oryx', principal='oryx/127.7.67.253@KRBTEST.COM'} at 127.0.0.1:54042)
I20260806 18:30:07.119519 16422 tablet_copy_service.cc:161] P c6a620695abe4ec7b4577e405a25f199: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bec8fce3919c48a4b5d7e5cfb2a96b11 at {username='oryx', principal='oryx/127.7.67.253@KRBTEST.COM'} at 127.0.0.1:54042: session id = bec8fce3919c48a4b5d7e5cfb2a96b11-00000000000000000000000000000000
I20260806 18:30:07.121160 16422 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 18:30:07.123046 17027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 18:30:07.125715 17027 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: tablet copy: Starting download of 0 data blocks...
I20260806 18:30:07.125932 17027 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: tablet copy: Starting download of 1 WAL segments...
I20260806 18:30:07.127970 17027 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 18:30:07.129825 16422 tablet_copy_service.cc:342] P c6a620695abe4ec7b4577e405a25f199: Request end of tablet copy session bec8fce3919c48a4b5d7e5cfb2a96b11-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.67.253@KRBTEST.COM'} at 127.0.0.1:54042
I20260806 18:30:07.129956 16422 tablet_copy_service.cc:433] P c6a620695abe4ec7b4577e405a25f199: ending tablet copy session bec8fce3919c48a4b5d7e5cfb2a96b11-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bec8fce3919c48a4b5d7e5cfb2a96b11
I20260806 18:30:07.137825 17027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:07.139536 17183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:07.139787 17181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:07.139947 17178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:07.147437 17027 server_base.cc:1034] running on GCE node
I20260806 18:30:07.147662 17027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:07.148100 17027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:07.149326 17027 hybrid_clock.cc:648] HybridClock initialized: now 1786041007149316 us; error 35 us; skew 500 ppm
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041007, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:07.151187 17027 init.cc:377] Logged in from keytab as oryx/127.7.67.253@KRBTEST.COM (short username oryx)
I20260806 18:30:07.151747 17027 webserver.cc:460] Webserver started at http://127.7.67.253:38629/ using document root <none> and password file <none>
I20260806 18:30:07.151945 17027 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:07.151985 17027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:07.153394 17027 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:30:07.154489 17190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:07.154753 17027 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:30:07.154837 17027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/wal
uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11"
format_stamp: "Formatted at 2026-08-06 18:30:06 on dist-test-slave-5bn2"
server_key: "5bf2a32e4a323b0c02d89bfb4fc6fde1"
server_key_iv: "4d5394a6f2506141e24b57443579e552"
server_key_version: "encryptionkey@0"
I20260806 18:30:07.154949 17027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:07.165755 17027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:07.167934 17027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:07.173159 17027 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.253:38821
I20260806 18:30:07.173346 17252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.253:38821 every 8 connection(s)
I20260806 18:30:07.173802 17027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-1/data/info.pb
I20260806 18:30:07.175836 17253 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:30:07.176920 17253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: Bootstrap starting.
I20260806 18:30:07.187876 17253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:30:07.188241 17253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: Bootstrap complete.
I20260806 18:30:07.188870 17253 raft_consensus.cc:348] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: NON_VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: true } }
I20260806 18:30:07.189028 17253 raft_consensus.cc:729] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bec8fce3919c48a4b5d7e5cfb2a96b11, State: Initialized, Role: LEARNER
I20260806 18:30:07.189133 17253 consensus_queue.cc:260] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: NON_VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: true } }
I20260806 18:30:07.189515 17257 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: NON_VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: true } } }
I20260806 18:30:07.189625 17253 sys_catalog.cc:564] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:30:07.189733 17257 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [sys.catalog]: This master's current role is: LEARNER
I20260806 18:30:07.193585 17273 catalog_manager.cc:1259] Loaded cluster ID: 046fb588ad9f453898a7561cc299683e
I20260806 18:30:07.193660 17273 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: loading cluster ID for follower catalog manager: success
I20260806 18:30:07.195899 17273 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: acquiring CA information for follower catalog manager: success
I20260806 18:30:07.197463 17273 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 18:30:07.249584 17177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:07.249869 17177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:07.249930 17177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:07.253963 17177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:30:07.254030 17177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:07.254058 17177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:07.254079 17177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:30:07.254108 17177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:30:07.259712 17177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:36393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:39723,127.7.67.253:38821,127.7.67.252:40865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.252.keytab
--principal=oryx/127.7.67.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.252:40865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.67.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:07.260829 17177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:07.261847 17177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:07.270237 17284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:07.270255 17279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:07.270255 17281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:07.270480 17177 server_base.cc:1034] running on GCE node
I20260806 18:30:07.271196 17177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:07.272294 17177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:07.273455 17177 hybrid_clock.cc:648] HybridClock initialized: now 1786041007273437 us; error 30 us; skew 500 ppm
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041007, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:07.276185 17177 init.cc:377] Logged in from keytab as oryx/127.7.67.252@KRBTEST.COM (short username oryx)
I20260806 18:30:07.277102 17177 webserver.cc:460] Webserver started at http://127.7.67.252:38253/ using document root <none> and password file <none>
I20260806 18:30:07.277650 17177 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:07.277711 17177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:07.277909 17177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:07.280014 17177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/data/instance:
uuid: "7f88490fdc154ddbbbe5a0f3a68c2227"
format_stamp: "Formatted at 2026-08-06 18:30:07 on dist-test-slave-5bn2"
server_key: "9d926f175cfe8dcf97e38ec7b657bc15"
server_key_iv: "02fde3627008e42e4fb1e56e2d329df0"
server_key_version: "encryptionkey@0"
I20260806 18:30:07.280650 17177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/wal/instance:
uuid: "7f88490fdc154ddbbbe5a0f3a68c2227"
format_stamp: "Formatted at 2026-08-06 18:30:07 on dist-test-slave-5bn2"
server_key: "9d926f175cfe8dcf97e38ec7b657bc15"
server_key_iv: "02fde3627008e42e4fb1e56e2d329df0"
server_key_version: "encryptionkey@0"
I20260806 18:30:07.284391 17177 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:30:07.286921 17294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:07.288215 17177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:30:07.288321 17177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/wal
uuid: "7f88490fdc154ddbbbe5a0f3a68c2227"
format_stamp: "Formatted at 2026-08-06 18:30:07 on dist-test-slave-5bn2"
server_key: "9d926f175cfe8dcf97e38ec7b657bc15"
server_key_iv: "02fde3627008e42e4fb1e56e2d329df0"
server_key_version: "encryptionkey@0"
I20260806 18:30:07.288412 17177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:07.310105 17177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:07.313601 17177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:07.313796 17177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:07.320858 17177 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.252:40865
I20260806 18:30:07.321144 17365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.252:40865 every 8 connection(s)
I20260806 18:30:07.321874 17177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/data/info.pb
I20260806 18:30:07.323670 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 17177
I20260806 18:30:07.325001 17366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:07.329682 17366 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:30:07.332685 16656 heartbeater.cc:503] Master 127.7.67.254:39723 was elected leader, sending a full tablet report...
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.67.252@KRBTEST.COM
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041007, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.252@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM
I20260806 18:30:07.338586 17368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.325240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40192 (local address 127.7.67.252:40865)
0806 18:30:07.325650 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.325671 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.325714 (+ 43us) server_negotiation.cc:407] Connection header received
0806 18:30:07.326605 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.326629 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.327089 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.327447 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.328103 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.329568 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.330420 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.331115 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.331250 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.333941 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.333946 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.333955 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.333981 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.335655 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.336151 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.336156 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.336162 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.336301 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.336604 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.336606 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.336607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.337012 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.337169 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.337461 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.337722 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":115,"threads_started":1}
I20260806 18:30:07.341797 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.331727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54044 (local address 127.7.67.254:39723)
0806 18:30:07.331865 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.331869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.333099 (+ 1230us) server_negotiation.cc:407] Connection header received
0806 18:30:07.333237 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.333240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.333284 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.333355 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.334260 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.335040 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.336044 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.336330 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.336425 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.338712 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.338717 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.338720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.338923 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.340214 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.340706 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.340709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.340711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.340799 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.341062 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.341065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.341066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.341249 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.341335 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.341593 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.341684 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:30:07.341840 17369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.331582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54044)
0806 18:30:07.332946 (+ 1364us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.332998 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.333132 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.333508 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.333515 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.333985 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.334144 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.334151 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.335156 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.335159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.335923 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.335935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.336432 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.336436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.336562 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.337077 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.337094 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.338565 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.340315 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.340322 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.340326 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.340611 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.340877 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.340880 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.340881 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.340958 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.341339 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.341344 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.341436 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.341549 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.341686 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1194,"thread_start_us":174,"threads_started":1}
I20260806 18:30:07.342772 17366 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041007, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.252@KRBTEST.COM for oryx/127.7.67.253@KRBTEST.COM
I20260806 18:30:07.350390 17369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.343237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:38821 (local address 127.0.0.1:41590)
0806 18:30:07.343343 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.343355 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.343416 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.343843 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.343845 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.343984 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.344088 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.344094 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.344959 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.344962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.345475 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.345483 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.345903 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.345906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.346005 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.346354 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.346364 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.347519 (+ 1155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.349031 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.349034 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.349036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.349224 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.349484 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.349487 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.349489 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.349565 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.349911 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.349914 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.350007 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.350103 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.350229 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 18:30:07.350419 17370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.343372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41590 (local address 127.7.67.253:38821)
0806 18:30:07.343646 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.343650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.343663 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:30:07.343698 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.343700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.343737 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.343813 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.344199 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.344860 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.345582 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.345807 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.345893 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.347649 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.347652 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.347654 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.347676 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.348902 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.349324 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.349327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.349328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.349404 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.349645 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.349648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.349650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.349821 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.349905 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.350166 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.350277 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":119,"threads_started":1}
I20260806 18:30:07.351194 17366 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041007, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.252@KRBTEST.COM for oryx/127.7.67.252@KRBTEST.COM
I20260806 18:30:07.358381 17369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.351616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:40865 (local address 127.0.0.1:40200)
0806 18:30:07.351719 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.351733 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.351801 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.351979 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.351981 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.352125 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.352186 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.352191 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.353383 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.353386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.353866 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.353873 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.354262 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.354266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.354390 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.354727 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.354736 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.355919 (+ 1183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.357248 (+ 1329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.357250 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.357251 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.357426 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.357641 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.357643 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.357644 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.357713 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.357981 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.357983 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.358064 (+ 81us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.358135 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.358236 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":32}
I20260806 18:30:07.358410 17368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.351701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40200 (local address 127.7.67.252:40865)
0806 18:30:07.351780 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.351784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.351795 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:30:07.351865 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.351868 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.351903 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.351957 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.352281 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.353267 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.353965 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.354174 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.354255 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.356033 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.356036 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.356038 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.356059 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.357154 (+ 1095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.357507 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.357509 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.357510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.357577 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.357773 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.357774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.357775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.357919 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.357987 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.358180 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.358287 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":28}
I20260806 18:30:07.361255 17366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: Bootstrap starting.
I20260806 18:30:07.363678 17366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:07.364398 17366 log.cc:824] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:07.366290 17366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: No bootstrap required, opened a new log
I20260806 18:30:07.369524 17366 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } }
I20260806 18:30:07.369743 17366 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:07.369781 17366 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f88490fdc154ddbbbe5a0f3a68c2227, State: Initialized, Role: FOLLOWER
I20260806 18:30:07.370190 17366 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } }
I20260806 18:30:07.370708 17372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } } }
I20260806 18:30:07.370903 17372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:30:07.371430 17366 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 18:30:07.376351 17385 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:30:07.376436 17385 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:30:07.380115 17177 master_runner.cc:428] Detected that this master 7f88490fdc154ddbbbe5a0f3a68c2227 is joining an existing cluster
I20260806 18:30:07.380173 17177 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.67.252:40865
I20260806 18:30:07.391491 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.382332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54060 (local address 127.7.67.254:39723)
0806 18:30:07.382586 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.382590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.383096 (+ 506us) server_negotiation.cc:407] Connection header received
0806 18:30:07.383306 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.383310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.383366 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.383420 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.384220 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.384989 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.386224 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.386562 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.386635 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.388025 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.388029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.388032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.388063 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.389572 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.390048 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.390052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.390054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.390164 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.390403 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.390406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.390408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.390645 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.390767 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.391150 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.391285 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260806 18:30:07.391666 17395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.382168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54060)
0806 18:30:07.383010 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.383025 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.383173 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.383579 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.383581 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.383880 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.384099 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.384106 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.385110 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.385113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.386080 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.386091 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.386820 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.386823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.386939 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.387452 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.387466 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.387869 (+ 403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.389682 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.389685 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.389686 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.389935 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.390250 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.390252 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.390253 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.390331 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.390778 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.390781 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.390913 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.391237 (+ 324us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.391457 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"mutex_wait_us":48,"spinlock_wait_cycles":1792,"thread_start_us":122,"threads_started":1}
I20260806 18:30:07.392869 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 16478
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041005, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.254@KRBTEST.COM for oryx/127.7.67.252@KRBTEST.COM
I20260806 18:30:07.411733 17368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.383067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40204 (local address 127.7.67.252:40865)
0806 18:30:07.383161 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.383165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.383352 (+ 187us) server_negotiation.cc:407] Connection header received
0806 18:30:07.383472 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.383474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.383507 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.383549 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.384180 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.384952 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.386415 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.386691 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.386752 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.408075 (+ 21323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.408079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.408081 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.408102 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.409860 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.410286 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.410289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.410291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.410360 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.410597 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.410601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.410602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.410782 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.410845 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.411421 (+ 576us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.411566 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:30:07.412170 17396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.382764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:40865 (local address 127.0.0.1:40204)
0806 18:30:07.383272 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.383285 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.383395 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.383660 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.383662 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.383906 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.384061 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.384070 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.385073 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.385076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.386320 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.386329 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.386898 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.386901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.386991 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.407453 (+ 20462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.407470 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.407942 (+ 472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.409966 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.409969 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.409971 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.410201 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.410453 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.410455 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.410457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.410527 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.411133 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.411136 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.411240 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.411874 (+ 634us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.412007 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":7,"spinlock_wait_cycles":6656,"thread_start_us":128,"threads_started":1}
I20260806 18:30:07.412858 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:45973
--local_ip_for_outbound_sockets=127.7.67.193
--tserver_master_addrs=127.7.67.254:39723
--webserver_port=37127
--webserver_interface=127.7.67.193
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.193.keytab
--principal=oryx/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:36393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.67.254:39723,127.7.67.253:38821
--tserver_master_addrs=127.7.67.254:39723,127.7.67.253:38821,127.7.67.252:40865 with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:30:07.414832 17394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.382698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:38821 (local address 127.0.0.1:41598)
0806 18:30:07.383096 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.383111 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.383256 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.383674 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.383677 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.383967 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.384165 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.384173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.385261 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.385264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.386170 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.386181 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.386985 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.386988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.387087 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.407888 (+ 20801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.407917 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.408665 (+ 748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.412559 (+ 3894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.412562 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.412564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.412791 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.413775 (+ 984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.413778 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.413780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.413869 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.414317 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.414320 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.414410 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.414501 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.414651 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":34,"spinlock_wait_cycles":4352,"thread_start_us":96,"threads_started":1}
I20260806 18:30:07.415664 17370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.382315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41598 (local address 127.7.67.253:38821)
0806 18:30:07.382482 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.382486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.383184 (+ 698us) server_negotiation.cc:407] Connection header received
0806 18:30:07.383390 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.383393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.383453 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.383518 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.384308 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.385113 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.386321 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.386698 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.386776 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.408851 (+ 22075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.408856 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.408859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.408891 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.411973 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.413559 (+ 1586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.413562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.413564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.413663 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.413964 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.413967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.413969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.414187 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.414274 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.415355 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.415497 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 18:30:07.416316 17153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.400069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:40865 (local address 127.0.0.1:40206)
0806 18:30:07.400233 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.400253 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.400369 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.404118 (+ 3749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.404121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.404348 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.404478 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.404485 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.405724 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.405728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.406540 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.406554 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.407374 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.407378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.407492 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.409193 (+ 1701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.409208 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.411187 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.414678 (+ 3491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.414681 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.414683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.414993 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.415297 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.415300 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.415302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.415389 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.415809 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.415812 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.415914 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.415999 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.416164 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 18:30:07.416937 17397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.400289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40206 (local address 127.7.67.252:40865)
0806 18:30:07.403764 (+ 3475us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.403773 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.403791 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:30:07.403878 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.403882 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.403947 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.404609 (+ 662us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.404744 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.405586 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.406669 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.407209 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.408412 (+ 1203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.412944 (+ 4532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.412948 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.412950 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.412974 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.414551 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.415116 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.415119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.415121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.415201 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.415506 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.415509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.415511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.415699 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.416603 (+ 904us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.416677 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.416788 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3390,"thread_start_us":284,"threads_started":1}
I20260806 18:30:07.417865 16396 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.67.252:40865: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227"
member_type: NON_VOTER
last_known_addr {
host: "127.7.67.252"
port: 40865
}
attrs {
promote: true
}
}
dest_uuid: "c6a620695abe4ec7b4577e405a25f199"
cas_config_opid_index: 5
I20260806 18:30:07.418613 16396 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: NON_VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: true } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: NON_VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: true } }
I20260806 18:30:07.419857 17156 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 7f88490fdc154ddbbbe5a0f3a68c2227 (127.7.67.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: NON_VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: true } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: NON_VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: true } } }
I20260806 18:30:07.420439 17332 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:07.421371 17156 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.67.252:40865
I20260806 18:30:07.421613 17156 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c6a620695abe4ec7b4577e405a25f199" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: NON_VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: true } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: NON_VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: true } } }
I20260806 18:30:07.421717 17156 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:07.421938 17332 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 1 FOLLOWER]: Refusing update from remote peer c6a620695abe4ec7b4577e405a25f199: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 18:30:07.422173 17177 master.cc:537] Master@127.7.67.252:40865 shutting down...
I20260806 18:30:07.422582 17158 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: NON_VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:30:07.424535 16373 consensus_peers.cc:487] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 -> Peer 7f88490fdc154ddbbbe5a0f3a68c2227 (127.7.67.252:40865): Couldn't send request to peer 7f88490fdc154ddbbbe5a0f3a68c2227. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 18:30:07.425212 17153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.419898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:38821 (local address 127.0.0.1:41606)
0806 18:30:07.420027 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.420042 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.420120 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.420742 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.420745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.420765 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:07.420854 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.420861 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.422155 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.422158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.423578 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.423590 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.424805 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.424808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.424902 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.424949 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.425017 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.425086 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 18:30:07.425613 17370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.420143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41606 (local address 127.7.67.253:38821)
0806 18:30:07.420343 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.420347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.420359 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:30:07.420400 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.420403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.420460 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.420567 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:07.420976 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.421929 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.423714 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.424468 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.424580 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.424741 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.425404 (+ 663us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.425486 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260806 18:30:07.426164 17224 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 1 LEARNER]: Refusing update from remote peer c6a620695abe4ec7b4577e405a25f199: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 18:30:07.426614 17158 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: NON_VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 18:30:07.427745 17177 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:30:07.428033 17177 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:07.428234 17177 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: stopping tablet replica
I20260806 18:30:07.428716 17224 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 7f88490fdc154ddbbbe5a0f3a68c2227 (127.7.67.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: NON_VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: true } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: NON_VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: true } } }
I20260806 18:30:07.430717 17400 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6a620695abe4ec7b4577e405a25f199" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: NON_VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: true } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: NON_VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: true } } }
I20260806 18:30:07.430837 17400 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [sys.catalog]: This master's current role is: LEARNER
I20260806 18:30:07.431207 17257 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6a620695abe4ec7b4577e405a25f199. Latest consensus state: current_term: 1 leader_uuid: "c6a620695abe4ec7b4577e405a25f199" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: NON_VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: true } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: NON_VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: true } } }
I20260806 18:30:07.431320 17257 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [sys.catalog]: This master's current role is: LEARNER
I20260806 18:30:07.432969 17156 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199: attempting to promote NON_VOTER bec8fce3919c48a4b5d7e5cfb2a96b11 to VOTER
I20260806 18:30:07.433426 17156 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: NON_VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: true } }
I20260806 18:30:07.434691 17224 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 1 LEARNER]: Refusing update from remote peer c6a620695abe4ec7b4577e405a25f199: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 18:30:07.435130 17158 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260806 18:30:07.435642 16373 consensus_peers.cc:487] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 -> Peer 7f88490fdc154ddbbbe5a0f3a68c2227 (127.7.67.252:40865): Couldn't send request to peer 7f88490fdc154ddbbbe5a0f3a68c2227. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 18:30:07.437800 17158 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bec8fce3919c48a4b5d7e5cfb2a96b11 (127.7.67.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: NON_VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: true } } }
I20260806 18:30:07.438342 17224 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bec8fce3919c48a4b5d7e5cfb2a96b11 (127.7.67.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: NON_VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: true } } }
I20260806 18:30:07.439654 17158 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c6a620695abe4ec7b4577e405a25f199" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: NON_VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: true } } }
I20260806 18:30:07.439755 17158 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:07.439863 17158 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6a620695abe4ec7b4577e405a25f199" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: NON_VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: true } } }
I20260806 18:30:07.439929 17158 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:07.440104 17257 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6a620695abe4ec7b4577e405a25f199" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: NON_VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: true } } }
I20260806 18:30:07.440208 17257 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:30:07.448889 17177 master.cc:552] Master@127.7.67.252:40865 shutdown complete.
I20260806 18:30:07.448958 17177 master_runner.cc:305] Clearing existing system tablet
I20260806 18:30:07.449208 17177 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:07.449268 17177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:07.450616 17177 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 18:30:07.451548 17404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:07.451782 17177 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:07.451916 17177 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:30:07.451957 17177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/wal
uuid: "7f88490fdc154ddbbbe5a0f3a68c2227"
format_stamp: "Formatted at 2026-08-06 18:30:07 on dist-test-slave-5bn2"
server_key: "9d926f175cfe8dcf97e38ec7b657bc15"
server_key_iv: "02fde3627008e42e4fb1e56e2d329df0"
server_key_version: "encryptionkey@0"
I20260806 18:30:07.454088 17177 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:07.456629 17177 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 18:30:07.456732 17177 log.cc:1196] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 18:30:07.456974 17177 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: Deleting consensus metadata
I20260806 18:30:07.457144 17177 master_runner.cc:315] Copying system tablet from 127.7.67.254:39723
I20260806 18:30:07.458376 17177 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: tablet copy: Beginning tablet copy session from remote peer at address 127.7.67.254:39723
I20260806 18:30:07.471504 17413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.458963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54064)
0806 18:30:07.463062 (+ 4099us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.463079 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.463219 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.463529 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.463533 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.463764 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.463903 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.463911 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.464980 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.464983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.465930 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.465940 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.466662 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.466666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.466767 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.467374 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.467389 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.467838 (+ 449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.469776 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.469779 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.469781 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.470038 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.470339 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.470342 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.470344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.470454 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.470909 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.470912 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.471019 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.471127 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.471309 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4011,"thread_start_us":89,"threads_started":1}
I20260806 18:30:07.471535 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.459069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54064 (local address 127.7.67.254:39723)
0806 18:30:07.459201 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.459206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.463184 (+ 3978us) server_negotiation.cc:407] Connection header received
0806 18:30:07.463339 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.463343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.463404 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.463495 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.464032 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.464860 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.466060 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.466399 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.466514 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.467998 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.468002 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.468005 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.468035 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.469650 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.470148 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.470152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.470153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.470246 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.470544 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.470547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.470549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.470766 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.470892 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.471190 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.471359 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:30:07.472280 16422 tablet_copy_service.cc:140] P c6a620695abe4ec7b4577e405a25f199: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7f88490fdc154ddbbbe5a0f3a68c2227 ({username='oryx', principal='oryx/127.7.67.252@KRBTEST.COM'} at 127.0.0.1:54064)
I20260806 18:30:07.472378 16422 tablet_copy_service.cc:161] P c6a620695abe4ec7b4577e405a25f199: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7f88490fdc154ddbbbe5a0f3a68c2227 at {username='oryx', principal='oryx/127.7.67.252@KRBTEST.COM'} at 127.0.0.1:54064: session id = 7f88490fdc154ddbbbe5a0f3a68c2227-00000000000000000000000000000000
I20260806 18:30:07.473479 16422 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 18:30:07.474839 17177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 18:30:07.477269 17177 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: tablet copy: Starting download of 0 data blocks...
I20260806 18:30:07.477458 17177 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: tablet copy: Starting download of 1 WAL segments...
I20260806 18:30:07.478952 17177 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 18:30:07.480782 16422 tablet_copy_service.cc:342] P c6a620695abe4ec7b4577e405a25f199: Request end of tablet copy session 7f88490fdc154ddbbbe5a0f3a68c2227-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.67.252@KRBTEST.COM'} at 127.0.0.1:54064
I20260806 18:30:07.480882 16422 tablet_copy_service.cc:433] P c6a620695abe4ec7b4577e405a25f199: ending tablet copy session 7f88490fdc154ddbbbe5a0f3a68c2227-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7f88490fdc154ddbbbe5a0f3a68c2227
I20260806 18:30:07.488469 17177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:07.490114 17415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:07.495885 17416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:07.495918 17421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:07.499019 17177 server_base.cc:1034] running on GCE node
I20260806 18:30:07.499243 17177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:07.499577 17177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:07.500730 17177 hybrid_clock.cc:648] HybridClock initialized: now 1786041007500716 us; error 34 us; skew 500 ppm
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041007, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:07.502413 17177 init.cc:377] Logged in from keytab as oryx/127.7.67.252@KRBTEST.COM (short username oryx)
I20260806 18:30:07.502951 17177 webserver.cc:460] Webserver started at http://127.7.67.252:42365/ using document root <none> and password file <none>
I20260806 18:30:07.503152 17177 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:07.503196 17177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:07.504638 17177 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 18:30:07.505779 17429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:07.506034 17177 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 18:30:07.506116 17177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/wal
uuid: "7f88490fdc154ddbbbe5a0f3a68c2227"
format_stamp: "Formatted at 2026-08-06 18:30:07 on dist-test-slave-5bn2"
server_key: "9d926f175cfe8dcf97e38ec7b657bc15"
server_key_iv: "02fde3627008e42e4fb1e56e2d329df0"
server_key_version: "encryptionkey@0"
I20260806 18:30:07.506197 17177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:07.521556 17177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:07.523828 17177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:07.528499 17177 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.252:40865
I20260806 18:30:07.528546 17502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.252:40865 every 8 connection(s)
I20260806 18:30:07.529070 17177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/master-2/data/info.pb
I20260806 18:30:07.531175 17503 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:30:07.532313 17503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: Bootstrap starting.
I20260806 18:30:07.544497 17503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:30:07.544807 17503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: Bootstrap complete.
I20260806 18:30:07.545428 17503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: NON_VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: true } }
I20260806 18:30:07.545583 17503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7f88490fdc154ddbbbe5a0f3a68c2227, State: Initialized, Role: LEARNER
I20260806 18:30:07.545699 17503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: NON_VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: true } }
I20260806 18:30:07.546056 17506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: NON_VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: true } } }
I20260806 18:30:07.546190 17503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:30:07.546274 17506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [sys.catalog]: This master's current role is: LEARNER
I20260806 18:30:07.550613 17522 catalog_manager.cc:1259] Loaded cluster ID: 046fb588ad9f453898a7561cc299683e
I20260806 18:30:07.550766 17522 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: loading cluster ID for follower catalog manager: success
I20260806 18:30:07.554574 17522 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: acquiring CA information for follower catalog manager: success
I20260806 18:30:07.556301 17522 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:30:07.560781 16839 heartbeater.cc:503] Master 127.7.67.254:39723 was elected leader, sending a full tablet report...
W20260806 18:30:07.560817 17398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:07.561123 17398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:07.561192 17398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:07.566316 17398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:07.566398 17398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:07.566488 17398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:30:07.573406 17398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:36393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.193.keytab
--principal=oryx/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:45973
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=37127
--tserver_master_addrs=127.7.67.254:39723,127.7.67.253:38821,127.7.67.252:40865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:07.574900 17398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:07.576305 17398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:07.588322 17531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:07.588297 17532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:07.588312 17534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:07.588721 17398 server_base.cc:1034] running on GCE node
I20260806 18:30:07.589175 17398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:07.589982 17398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:07.591142 17398 hybrid_clock.cc:648] HybridClock initialized: now 1786041007591070 us; error 91 us; skew 500 ppm
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041007, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:07.594270 17398 init.cc:377] Logged in from keytab as oryx/127.7.67.193@KRBTEST.COM (short username oryx)
I20260806 18:30:07.595360 17398 webserver.cc:460] Webserver started at http://127.7.67.193:37127/ using document root <none> and password file <none>
I20260806 18:30:07.595928 17398 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:07.596015 17398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:07.600531 17398 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:30:07.602885 17545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:07.604296 17398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:30:07.604396 17398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "04c0f654471b4bf19e477ac1f30b535b"
format_stamp: "Formatted at 2026-08-06 18:30:06 on dist-test-slave-5bn2"
server_key: "a7b0867163bdfe558058dfd73d69bf12"
server_key_iv: "bea5ca5911df1b8655ef291f1ee767ed"
server_key_version: "encryptionkey@0"
I20260806 18:30:07.604928 17398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:07.627427 17398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:07.630191 17398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:07.630369 17398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:07.631001 17398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:07.631881 17398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:07.631938 17398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:07.631994 17398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:07.632021 17398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:07.643074 17398 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:45973
I20260806 18:30:07.643080 17700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:45973 every 8 connection(s)
I20260806 18:30:07.644129 17398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:30:07.644598 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 17398
I20260806 18:30:07.644815 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 16666
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041007, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.252@KRBTEST.COM
I20260806 18:30:07.665517 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.647438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:48837 (local address 127.7.67.252:40865)
0806 18:30:07.648735 (+ 1297us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.648740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.649920 (+ 1180us) server_negotiation.cc:407] Connection header received
0806 18:30:07.650629 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.650632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.650687 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.650754 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.652285 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.653195 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.655813 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.656393 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.656496 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.660309 (+ 3813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.660314 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.660316 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.660350 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.663004 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.663647 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.663651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.663653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.663748 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.664161 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.664164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.664165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.664398 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.664485 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.665188 (+ 703us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.665330 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1219,"thread_start_us":899,"threads_started":1}
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041007, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.253@KRBTEST.COM
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041007, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM
I20260806 18:30:07.667443 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:32907
--local_ip_for_outbound_sockets=127.7.67.194
--tserver_master_addrs=127.7.67.254:39723
--webserver_port=45123
--webserver_interface=127.7.67.194
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.194.keytab
--principal=oryx/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:36393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.67.254:39723,127.7.67.253:38821
--tserver_master_addrs=127.7.67.254:39723,127.7.67.253:38821,127.7.67.252:40865 with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:30:07.667577 17706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.646833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:40865 (local address 127.7.67.193:48837)
0806 18:30:07.649134 (+ 2301us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.649214 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.650307 (+ 1093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.651065 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.651074 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.651520 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.652130 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.652148 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.653345 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.653348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.655630 (+ 2282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.655646 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.656764 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.656769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.656962 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.658269 (+ 1307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.658289 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.660091 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.663146 (+ 3055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.663152 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.663171 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.663538 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.663848 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.663851 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.663852 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.664055 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.664555 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.664561 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.664875 (+ 314us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.665766 (+ 891us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.666142 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1997,"thread_start_us":2442,"threads_started":1}
I20260806 18:30:07.668830 17702 heartbeater.cc:347] Connected to a master server at 127.7.67.252:40865
I20260806 18:30:07.669085 17702 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:07.669648 17702 heartbeater.cc:511] Master 127.7.67.252:40865 requested a full tablet report, sending...
I20260806 18:30:07.671379 17452 ts_manager.cc:194] Registered new tserver with Master: 04c0f654471b4bf19e477ac1f30b535b (127.7.67.193:45973)
I20260806 18:30:07.673118 17710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.657369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:38821 (local address 127.7.67.193:52765)
0806 18:30:07.660319 (+ 2950us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.660332 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.660467 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.660842 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.660844 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.661089 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.661260 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.661268 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.662483 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.662486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.663466 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.663477 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.664166 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.664169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.664324 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.664861 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.664875 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.666893 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.671464 (+ 4571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.671467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.671469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.671720 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.672059 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.672062 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.672064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.672150 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.672619 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.672622 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.672703 (+ 81us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.672779 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.672919 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2881,"mutex_wait_us":57,"thread_start_us":421,"threads_started":1}
I20260806 18:30:07.673131 17370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.659887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:52765 (local address 127.7.67.253:38821)
0806 18:30:07.660027 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.660032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.660400 (+ 368us) server_negotiation.cc:407] Connection header received
0806 18:30:07.660593 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.660597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.660657 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.660708 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.661379 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.662151 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.663615 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.663933 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.664007 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.669480 (+ 5473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.669485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.669488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.669519 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.671342 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.671844 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.671847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.671849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.671952 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.672244 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.672248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.672249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.672473 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.672568 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.672874 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.673000 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:30:07.673782 17703 heartbeater.cc:347] Connected to a master server at 127.7.67.253:38821
I20260806 18:30:07.673889 17703 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:07.674067 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.647279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:47359 (local address 127.7.67.254:39723)
0806 18:30:07.648567 (+ 1288us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.648573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.658128 (+ 9555us) server_negotiation.cc:407] Connection header received
0806 18:30:07.660223 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.660227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.660284 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.660334 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.661584 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.662336 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.663631 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.663941 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.664014 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.669489 (+ 5475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.669494 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.669512 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.669542 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.671744 (+ 2202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.672209 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.672212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.672214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.672306 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.672620 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.672624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.672625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.672848 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.672937 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.673793 (+ 856us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.673924 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1175}
I20260806 18:30:07.674081 17703 heartbeater.cc:511] Master 127.7.67.253:38821 requested a full tablet report, sending...
I20260806 18:30:07.674101 17711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.646833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.193:47359)
0806 18:30:07.658022 (+ 11189us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.658047 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.660072 (+ 2025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.660475 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.660477 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.661052 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.661431 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.661442 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.662470 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.662474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.663485 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.663504 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.664178 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.664181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.664283 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.664970 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.664984 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.669301 (+ 4317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.671843 (+ 2542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.671846 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.671848 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.672098 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.672419 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.672421 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.672423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.672503 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.672984 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.672987 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.673088 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.673600 (+ 512us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.673749 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11067,"mutex_wait_us":85,"spinlock_wait_cycles":101504,"thread_start_us":1333,"threads_started":1}
I20260806 18:30:07.675158 17701 heartbeater.cc:347] Connected to a master server at 127.7.67.254:39723
I20260806 18:30:07.675181 17212 ts_manager.cc:194] Registered new tserver with Master: 04c0f654471b4bf19e477ac1f30b535b (127.7.67.193:45973)
I20260806 18:30:07.675252 17701 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:07.675421 17701 heartbeater.cc:511] Master 127.7.67.254:39723 requested a full tablet report, sending...
I20260806 18:30:07.676159 16396 ts_manager.cc:194] Re-registered known tserver with Master: 04c0f654471b4bf19e477ac1f30b535b (127.7.67.193:45973)
I20260806 18:30:07.676733 16396 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:47359
I20260806 18:30:07.781467 17021 heartbeater.cc:503] Master 127.7.67.254:39723 was elected leader, sending a full tablet report...
W20260806 18:30:07.785173 17712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:07.785470 17712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:07.785542 17712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:07.789495 17712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:07.789549 17712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:07.789623 17712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:30:07.794953 17712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:36393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.194.keytab
--principal=oryx/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:32907
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=45123
--tserver_master_addrs=127.7.67.254:39723,127.7.67.253:38821,127.7.67.252:40865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:07.796000 17712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:07.796993 17712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:07.806358 17720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:07.806339 17719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:07.806403 17723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:07.806733 17712 server_base.cc:1034] running on GCE node
I20260806 18:30:07.807168 17712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:07.807793 17712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:07.808970 17712 hybrid_clock.cc:648] HybridClock initialized: now 1786041007808940 us; error 38 us; skew 500 ppm
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041007, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:07.811792 17712 init.cc:377] Logged in from keytab as oryx/127.7.67.194@KRBTEST.COM (short username oryx)
I20260806 18:30:07.812631 17712 webserver.cc:460] Webserver started at http://127.7.67.194:45123/ using document root <none> and password file <none>
I20260806 18:30:07.813167 17712 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:07.813247 17712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:07.817109 17712 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:30:07.819067 17732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:07.820240 17712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:30:07.820348 17712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "9a703950a82f49cb8d102abf747e0f44"
format_stamp: "Formatted at 2026-08-06 18:30:06 on dist-test-slave-5bn2"
server_key: "09fd2f616ecde7e1199de99590f20894"
server_key_iv: "249913cacda9aebd0f57064283933fed"
server_key_version: "encryptionkey@0"
I20260806 18:30:07.820889 17712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:07.833570 17712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:07.836613 17712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:07.836778 17712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:07.837339 17712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:07.838294 17712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:07.838341 17712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:07.838388 17712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:07.838415 17712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:07.847566 17712 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:32907
I20260806 18:30:07.847584 17884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:32907 every 8 connection(s)
I20260806 18:30:07.848673 17712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:30:07.856559 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 17712
I20260806 18:30:07.856755 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 16847
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041007, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.253@KRBTEST.COM
I20260806 18:30:07.863593 17370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.850596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:49045 (local address 127.7.67.253:38821)
0806 18:30:07.850734 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.850739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.852799 (+ 2060us) server_negotiation.cc:407] Connection header received
0806 18:30:07.853434 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.853437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.853492 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.853570 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.854768 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.855612 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.856925 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.857193 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.857254 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.859922 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.859926 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.859928 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.859954 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.861436 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.862002 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.862006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.862008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.862098 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.862429 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.862432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.862433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.862643 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.862700 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.863315 (+ 615us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.863428 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:30:07.864967 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.851160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:38821 (local address 127.7.67.194:49045)
0806 18:30:07.852708 (+ 1548us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.852717 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.852963 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.853603 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.853606 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.854035 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.854614 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.854620 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.855757 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.855761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.856808 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.856814 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.857431 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.857434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.857606 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.858161 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.858179 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.859753 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.861578 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.861584 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.861599 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.861904 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.862189 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.862192 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.862194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.862346 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.862768 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.862775 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.863080 (+ 305us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.863745 (+ 665us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.864060 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1467,"mutex_wait_us":342,"spinlock_wait_cycles":11392,"thread_start_us":92,"threads_started":1}
I20260806 18:30:07.870599 17888 heartbeater.cc:347] Connected to a master server at 127.7.67.253:38821
I20260806 18:30:07.870906 17888 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:07.871551 17888 heartbeater.cc:511] Master 127.7.67.253:38821 requested a full tablet report, sending...
I20260806 18:30:07.873128 17212 ts_manager.cc:194] Registered new tserver with Master: 9a703950a82f49cb8d102abf747e0f44 (127.7.67.194:32907)
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041007, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.252@KRBTEST.COM
I20260806 18:30:07.877033 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.850912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:33847 (local address 127.7.67.252:40865)
0806 18:30:07.851185 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.851189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.852566 (+ 1377us) server_negotiation.cc:407] Connection header received
0806 18:30:07.853161 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.853165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.853222 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.853267 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.854809 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.855612 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.870751 (+ 15139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.871150 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.871239 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.873932 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.873936 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.873940 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.873970 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.875458 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.875858 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.875862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.875864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.875942 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.876212 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.876215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.876216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.876398 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.876477 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.876750 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.876875 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195}
I20260806 18:30:07.877043 17892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.851160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:40865 (local address 127.7.67.194:33847)
0806 18:30:07.852457 (+ 1297us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.852468 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.852962 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.853469 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.853476 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.854047 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.854609 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.854619 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.855757 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.855761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.856801 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.856813 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.871413 (+ 14600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.871416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.871519 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.872016 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.872029 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.873764 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.875571 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.875574 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.875576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.875767 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.876038 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.876041 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.876043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.876124 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.876506 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.876510 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.876599 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.876684 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.876832 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1213,"mutex_wait_us":152,"spinlock_wait_cycles":51712,"thread_start_us":149,"threads_started":1}
I20260806 18:30:07.881140 17153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.866067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:40865 (local address 127.0.0.1:40212)
0806 18:30:07.866314 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.866326 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.866396 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.877430 (+ 11034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.877434 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.877459 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:07.877569 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.877576 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.878276 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.878280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.879466 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.879482 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.880692 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.880697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.880795 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.880822 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.880895 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.880985 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260806 18:30:07.881685 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.866550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40212 (local address 127.7.67.252:40865)
0806 18:30:07.877196 (+ 10646us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.877199 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.877210 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:30:07.877244 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.877246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.877293 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.877377 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:07.877674 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.878166 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.879640 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.880312 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.880423 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.880613 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.881440 (+ 827us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.881549 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10584,"thread_start_us":504,"threads_started":1}
I20260806 18:30:07.887871 17466 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 18:30:07.888356 17506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6a620695abe4ec7b4577e405a25f199. Latest consensus state: current_term: 1 leader_uuid: "c6a620695abe4ec7b4577e405a25f199" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: NON_VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: true } } }
I20260806 18:30:07.888513 17506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [sys.catalog]: This master's current role is: LEARNER
I20260806 18:30:07.891580 17887 heartbeater.cc:347] Connected to a master server at 127.7.67.252:40865
I20260806 18:30:07.891713 17887 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:07.891978 17887 heartbeater.cc:511] Master 127.7.67.252:40865 requested a full tablet report, sending...
I20260806 18:30:07.892834 17452 ts_manager.cc:194] Registered new tserver with Master: 9a703950a82f49cb8d102abf747e0f44 (127.7.67.194:32907)
I20260806 18:30:07.894498 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:45549
--local_ip_for_outbound_sockets=127.7.67.195
--tserver_master_addrs=127.7.67.254:39723
--webserver_port=36673
--webserver_interface=127.7.67.195
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.195.keytab
--principal=oryx/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:36393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.67.254:39723,127.7.67.253:38821
--tserver_master_addrs=127.7.67.254:39723,127.7.67.253:38821,127.7.67.252:40865 with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:30:07 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041007, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM
I20260806 18:30:07.898095 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.850951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:46997 (local address 127.7.67.254:39723)
0806 18:30:07.851086 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:07.851090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:07.852305 (+ 1215us) server_negotiation.cc:407] Connection header received
0806 18:30:07.853152 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:07.853155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:07.853214 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:07.853286 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:07.854734 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.855340 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.856810 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.857142 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.857226 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.895151 (+ 37925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:07.895156 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:07.895159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:07.895192 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:07.896545 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.897020 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.897022 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.897023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.897091 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:07.897354 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:07.897356 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:07.897357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:07.897528 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:07.897596 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:07.897871 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 18:30:07.897975 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:30:07.898178 17893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:07.851160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.194:46997)
0806 18:30:07.852155 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:07.852195 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:07.852952 (+ 757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:07.853470 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:07.853477 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:07.854013 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:07.854601 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.854616 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.855489 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.855492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.856566 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:07.856578 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:07.892348 (+ 35770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:07.892351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:07.892476 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:07.893043 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:07.893058 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:07.894972 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:07.896661 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.896664 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.896666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.896921 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.897168 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:07.897170 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:07.897172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:07.897274 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:07.897615 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:07.897618 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:07.897730 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 18:30:07.897838 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 18:30:07.897987 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"mutex_wait_us":33,"thread_start_us":166,"threads_started":1}
I20260806 18:30:07.898810 17885 heartbeater.cc:347] Connected to a master server at 127.7.67.254:39723
I20260806 18:30:07.898936 17885 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:07.899139 17885 heartbeater.cc:511] Master 127.7.67.254:39723 requested a full tablet report, sending...
I20260806 18:30:07.899829 16396 ts_manager.cc:194] Re-registered known tserver with Master: 9a703950a82f49cb8d102abf747e0f44 (127.7.67.194:32907)
I20260806 18:30:07.900359 16396 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:46997
W20260806 18:30:08.011137 17896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:08.011411 17896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:08.011475 17896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:08.015532 17896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:08.015602 17896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:08.015681 17896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:30:08.021034 17896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:36393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.195.keytab
--principal=oryx/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:45549
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=36673
--tserver_master_addrs=127.7.67.254:39723,127.7.67.253:38821,127.7.67.252:40865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:08.022150 17896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:08.023336 17896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:08.033349 17905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:08.033385 17903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:08.033408 17902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:08.033718 17896 server_base.cc:1034] running on GCE node
I20260806 18:30:08.034235 17896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:08.034925 17896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:08.036089 17896 hybrid_clock.cc:648] HybridClock initialized: now 1786041008036075 us; error 26 us; skew 500 ppm
Aug 06 18:30:08 dist-test-slave-5bn2 krb5kdc[16334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041008, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:08.038811 17896 init.cc:377] Logged in from keytab as oryx/127.7.67.195@KRBTEST.COM (short username oryx)
I20260806 18:30:08.039742 17896 webserver.cc:460] Webserver started at http://127.7.67.195:36673/ using document root <none> and password file <none>
I20260806 18:30:08.040282 17896 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:08.040356 17896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:08.044375 17896 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:30:08.046486 17913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:08.047966 17896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:30:08.048097 17896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "09fb11cf02c9434fb56d3d390a1a3536"
format_stamp: "Formatted at 2026-08-06 18:30:06 on dist-test-slave-5bn2"
server_key: "d61a334f1aca2d66524ef57bb0475507"
server_key_iv: "1ded5a65224ea11ce6ef5a7b6d394877"
server_key_version: "encryptionkey@0"
I20260806 18:30:08.048718 17896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:08.062618 17896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:08.065677 17896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:08.065864 17896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:08.066490 17896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:08.067447 17896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:08.067507 17896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:08.067561 17896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:08.067588 17896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:08.081403 17896 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:45549
I20260806 18:30:08.081429 18064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:45549 every 8 connection(s)
I20260806 18:30:08.082499 17896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:30:08.092219 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 17896
Aug 06 18:30:08 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041008, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.252@KRBTEST.COM
Aug 06 18:30:08 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041008, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM
Aug 06 18:30:08 dist-test-slave-5bn2 krb5kdc[16334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041008, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.253@KRBTEST.COM
I20260806 18:30:08.100514 17895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.085227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:58165 (local address 127.7.67.252:40865)
0806 18:30:08.085387 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:08.085393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:08.086413 (+ 1020us) server_negotiation.cc:407] Connection header received
0806 18:30:08.087386 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:08.087390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:08.087454 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:08.087547 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:08.090195 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.090788 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.091978 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.092248 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.092340 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.096126 (+ 3786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:08.096130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:08.096133 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:08.096168 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:08.097930 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.098579 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.098583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.098584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.098684 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.099158 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.099162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.099164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.099388 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:08.099457 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:08.100181 (+ 724us) server_negotiation.cc:299] Negotiation successful
0806 18:30:08.100321 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:30:08.101835 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.085141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:40865 (local address 127.7.67.195:58165)
0806 18:30:08.086263 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:08.086307 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:08.087176 (+ 869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:08.087818 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:08.087825 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:08.088408 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:08.089025 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:08.089040 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.090935 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.090939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:08.091811 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:08.091822 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.092454 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.092459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:08.092622 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.093206 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:08.093230 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:08.095963 (+ 2733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:08.098071 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:08.098078 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:08.098098 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:08.098459 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:08.098794 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:08.098797 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:08.098799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:08.099033 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:08.099557 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:08.099563 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:08.099897 (+ 334us) client_negotiation.cc:769] Sending connection context
0806 18:30:08.100715 (+ 818us) client_negotiation.cc:240] Negotiation successful
0806 18:30:08.100859 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1033,"mutex_wait_us":349,"spinlock_wait_cycles":23040,"thread_start_us":99,"threads_started":1}
I20260806 18:30:08.102123 18071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.085141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.195:36141)
0806 18:30:08.086256 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:08.086310 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:08.087182 (+ 872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:08.087840 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:08.087843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:08.088408 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:08.089025 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:08.089040 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.090166 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.090170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:08.091831 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:08.091842 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.092674 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.092678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:08.092771 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.093309 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:08.093325 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:08.095777 (+ 2452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:08.098924 (+ 3147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:08.098927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:08.098929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:08.099180 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:08.099500 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:08.099503 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:08.099505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:08.099598 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:08.100059 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:08.100062 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:08.100153 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 18:30:08.100256 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 18:30:08.100620 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":897,"mutex_wait_us":170,"spinlock_wait_cycles":110080,"thread_start_us":146,"threads_started":1}
I20260806 18:30:08.103241 18067 heartbeater.cc:347] Connected to a master server at 127.7.67.252:40865
I20260806 18:30:08.103519 18067 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:08.104172 18067 heartbeater.cc:511] Master 127.7.67.252:40865 requested a full tablet report, sending...
I20260806 18:30:08.104418 18072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.085141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:38821 (local address 127.7.67.195:40457)
0806 18:30:08.086256 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:08.086310 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:08.087182 (+ 872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:08.087816 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:08.087824 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:08.088422 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:08.089022 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:08.089037 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.090166 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.090170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:08.091850 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:08.091857 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.094751 (+ 2894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.094756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:08.094879 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.095413 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:08.095427 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:08.096879 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:08.102642 (+ 5763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:08.102645 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:08.102647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:08.102939 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:08.103282 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:08.103286 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:08.103288 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:08.103383 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:08.103870 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:08.103873 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:08.103977 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 18:30:08.104087 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 18:30:08.104234 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":957,"mutex_wait_us":277,"spinlock_wait_cycles":56448,"thread_start_us":221,"threads_started":1}
I20260806 18:30:08.105000 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.084629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:36141 (local address 127.7.67.254:39723)
0806 18:30:08.084779 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:08.084784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:08.086416 (+ 1632us) server_negotiation.cc:407] Connection header received
0806 18:30:08.087430 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:08.087434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:08.087502 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:08.087679 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:08.089202 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.090012 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.091979 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.092298 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.092380 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.095976 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:08.095980 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:08.095983 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:08.096020 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:08.098384 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.099307 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.099310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.099312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.099399 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.099701 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.099704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.099706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.099913 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:08.100000 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:08.104732 (+ 4732us) server_negotiation.cc:299] Negotiation successful
0806 18:30:08.104847 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:30:08.105564 17370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.084631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:40457 (local address 127.7.67.253:38821)
0806 18:30:08.084780 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:08.084785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:08.086416 (+ 1631us) server_negotiation.cc:407] Connection header received
0806 18:30:08.087537 (+ 1121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:08.087540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:08.087594 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:08.087668 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:08.089176 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.090012 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.094236 (+ 4224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.094602 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.094717 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.097023 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:08.097026 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:08.097029 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:08.097066 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:08.098798 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.103063 (+ 4265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.103067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.103069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.103174 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.103490 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.103494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.103495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.103728 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:08.103819 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:08.105247 (+ 1428us) server_negotiation.cc:299] Negotiation successful
0806 18:30:08.105416 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:30:08.105865 17452 ts_manager.cc:194] Registered new tserver with Master: 09fb11cf02c9434fb56d3d390a1a3536 (127.7.67.195:45549)
I20260806 18:30:08.106240 18068 heartbeater.cc:347] Connected to a master server at 127.7.67.253:38821
I20260806 18:30:08.106334 18068 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:08.106510 18068 heartbeater.cc:511] Master 127.7.67.253:38821 requested a full tablet report, sending...
I20260806 18:30:08.106702 18065 heartbeater.cc:347] Connected to a master server at 127.7.67.254:39723
I20260806 18:30:08.106779 18065 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:08.106966 18065 heartbeater.cc:511] Master 127.7.67.254:39723 requested a full tablet report, sending...
I20260806 18:30:08.107225 18078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.094515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54070 (local address 127.7.67.254:39723)
0806 18:30:08.097108 (+ 2593us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:08.097112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:08.097124 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:30:08.097160 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:08.097163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:08.097208 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:08.098528 (+ 1320us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:08.098654 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.099412 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.100572 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.100899 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.101382 (+ 483us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.102643 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:08.102646 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:08.102649 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:08.102671 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:08.104583 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.105275 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.105278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.105280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.105367 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.105703 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.105705 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.105707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.105915 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:08.106613 (+ 698us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:08.106690 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 18:30:08.106808 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2516,"thread_start_us":84,"threads_started":1}
I20260806 18:30:08.107687 17212 ts_manager.cc:194] Registered new tserver with Master: 09fb11cf02c9434fb56d3d390a1a3536 (127.7.67.195:45549)
I20260806 18:30:08.108083 16395 ts_manager.cc:194] Re-registered known tserver with Master: 09fb11cf02c9434fb56d3d390a1a3536 (127.7.67.195:45549)
I20260806 18:30:08.108744 16395 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:36141
I20260806 18:30:08.111740 18081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.095398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41622 (local address 127.7.67.253:38821)
0806 18:30:08.095675 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:08.095680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:08.098493 (+ 2813us) server_negotiation.cc:407] Connection header received
0806 18:30:08.098539 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:08.098542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:08.098594 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:08.098672 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:08.099496 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.100211 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.102433 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.102766 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.102894 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.104678 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:08.104682 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:08.104685 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:08.104711 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:08.109806 (+ 5095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.110208 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.110211 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.110213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.110295 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.110541 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.110544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.110546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.110729 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:08.111385 (+ 656us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:08.111461 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 18:30:08.111583 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":92,"threads_started":1}
I20260806 18:30:08.113196 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.095211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40224 (local address 127.7.67.252:40865)
0806 18:30:08.095704 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:08.095708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:08.096605 (+ 897us) server_negotiation.cc:407] Connection header received
0806 18:30:08.097425 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:08.097429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:08.097478 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:08.097550 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:08.105934 (+ 8384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.106692 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.107767 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.108067 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.108144 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.109918 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:08.109922 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:08.109924 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:08.109950 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:08.111546 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.111935 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.111938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.111940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.112016 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.112326 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.112330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.112332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.112503 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:08.112606 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:08.112929 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 18:30:08.113037 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413}
I20260806 18:30:08.111156 16395 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:30:08.114781 16395 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:30:08.117287 17402 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6a620695abe4ec7b4577e405a25f199" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: NON_VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: true } } }
I20260806 18:30:08.117429 17402 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:08.121237 18085 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199: attempting to promote NON_VOTER 7f88490fdc154ddbbbe5a0f3a68c2227 to VOTER
I20260806 18:30:08.121703 18085 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } }
I20260806 18:30:08.122802 17224 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 1 FOLLOWER]: Refusing update from remote peer c6a620695abe4ec7b4577e405a25f199: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 18:30:08.122819 17466 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 1 LEARNER]: Refusing update from remote peer c6a620695abe4ec7b4577e405a25f199: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 18:30:08.123215 18089 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 18:30:08.123492 18085 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 18:30:08.125671 17402 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 7f88490fdc154ddbbbe5a0f3a68c2227 (127.7.67.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } } }
I20260806 18:30:08.125742 17466 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 7f88490fdc154ddbbbe5a0f3a68c2227 (127.7.67.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } } }
I20260806 18:30:08.127385 18085 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c6a620695abe4ec7b4577e405a25f199" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } } }
I20260806 18:30:08.127506 18085 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:08.127606 18085 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6a620695abe4ec7b4577e405a25f199" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } } }
I20260806 18:30:08.127686 18085 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:08.127655 18098 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6a620695abe4ec7b4577e405a25f199" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } } }
I20260806 18:30:08.127857 18098 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:08.127868 17506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6a620695abe4ec7b4577e405a25f199" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } } }
I20260806 18:30:08.127990 17506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:30:08.128260 17224 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 7f88490fdc154ddbbbe5a0f3a68c2227 (127.7.67.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } } }
I20260806 18:30:08.129838 18103 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6a620695abe4ec7b4577e405a25f199" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } } }
I20260806 18:30:08.130177 18103 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:30:08.137534 17153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.132216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:32907 (local address 127.0.0.1:37986)
0806 18:30:08.132378 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:08.132393 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:08.132473 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:08.133416 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:08.133419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:08.133440 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:08.133576 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:08.133584 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.135189 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.135193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:08.135988 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:08.136008 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.137145 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.137148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:08.137236 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.137264 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 18:30:08.137313 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 18:30:08.137383 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"spinlock_wait_cycles":3328}
I20260806 18:30:08.138109 18108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.132867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:45973 (local address 127.0.0.1:48616)
0806 18:30:08.133104 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:08.133117 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:08.133199 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:08.134274 (+ 1075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:08.134277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:08.134295 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:08.134393 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:08.134400 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.135838 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.135842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:08.136701 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:08.136712 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.137752 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.137756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:08.137846 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.137872 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:30:08.137910 (+ 38us) client_negotiation.cc:240] Negotiation successful
0806 18:30:08.137967 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":1,"thread_start_us":96,"threads_started":1}
I20260806 18:30:08.138329 18104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.132405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37986 (local address 127.7.67.194:32907)
0806 18:30:08.132863 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:08.132875 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:08.132899 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:30:08.132984 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:08.132989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:08.133239 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:08.133747 (+ 508us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:08.133912 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.135081 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.136129 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.137011 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.137842 (+ 831us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.138021 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:08.138081 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 18:30:08.138189 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":242,"threads_started":1}
I20260806 18:30:08.138697 18111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.133598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54826 (local address 127.7.67.195:45549)
0806 18:30:08.134019 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:08.134036 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:08.134070 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:30:08.134148 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:08.134153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:08.134333 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:08.134493 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:08.134677 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.135777 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.137141 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.137902 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.138003 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.138139 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:08.138455 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 18:30:08.138556 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":99,"threads_started":1}
I20260806 18:30:08.138830 18109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.133048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48616 (local address 127.7.67.193:45973)
0806 18:30:08.133775 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:08.133792 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:08.133815 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:30:08.133892 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:08.133897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:08.134128 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:08.134287 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:08.134522 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.135716 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.136849 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.137610 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.138379 (+ 769us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.138548 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:08.138598 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 18:30:08.138691 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":152,"threads_started":1}
I20260806 18:30:08.138720 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.133184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:45549 (local address 127.0.0.1:54826)
0806 18:30:08.133399 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:08.133412 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:08.133501 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:08.134468 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:08.134471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:08.134489 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:08.134578 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:08.134584 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.135893 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.135896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:08.136991 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:08.137003 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.138026 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.138030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:08.138109 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.138131 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:30:08.138511 (+ 380us) client_negotiation.cc:240] Negotiation successful
0806 18:30:08.138589 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"spinlock_wait_cycles":384,"thread_start_us":94,"threads_started":1}
I20260806 18:30:08.141955 17611 tablet_service.cc:1467] Processing CreateTablet for tablet aa05885955994ae3814408421673529c (DEFAULT_TABLE table=test-table [id=a4f2179fc6804efb88c20970a54c19f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:08.141952 17795 tablet_service.cc:1467] Processing CreateTablet for tablet aa05885955994ae3814408421673529c (DEFAULT_TABLE table=test-table [id=a4f2179fc6804efb88c20970a54c19f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:08.143256 17795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa05885955994ae3814408421673529c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:08.143256 17611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa05885955994ae3814408421673529c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:08.144922 17978 tablet_service.cc:1467] Processing CreateTablet for tablet aa05885955994ae3814408421673529c (DEFAULT_TABLE table=test-table [id=a4f2179fc6804efb88c20970a54c19f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:08.146054 17978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa05885955994ae3814408421673529c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:08.153014 18112 tablet_bootstrap.cc:492] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b: Bootstrap starting.
I20260806 18:30:08.153600 18114 tablet_bootstrap.cc:492] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536: Bootstrap starting.
I20260806 18:30:08.153975 18113 tablet_bootstrap.cc:492] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44: Bootstrap starting.
I20260806 18:30:08.155618 18112 tablet_bootstrap.cc:654] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:08.155892 18113 tablet_bootstrap.cc:654] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:08.156190 18114 tablet_bootstrap.cc:654] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:08.156592 18112 log.cc:824] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:08.156643 18113 log.cc:824] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:08.156962 18114 log.cc:824] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:08.158622 18113 tablet_bootstrap.cc:492] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44: No bootstrap required, opened a new log
I20260806 18:30:08.158629 18112 tablet_bootstrap.cc:492] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b: No bootstrap required, opened a new log
I20260806 18:30:08.158833 18113 ts_tablet_manager.cc:1397] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:30:08.158834 18112 ts_tablet_manager.cc:1397] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:30:08.160485 18114 tablet_bootstrap.cc:492] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536: No bootstrap required, opened a new log
I20260806 18:30:08.160684 18114 ts_tablet_manager.cc:1397] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 18:30:08.162694 18112 raft_consensus.cc:348] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } } peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } }
I20260806 18:30:08.162984 18112 raft_consensus.cc:374] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:08.163043 18112 raft_consensus.cc:729] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04c0f654471b4bf19e477ac1f30b535b, State: Initialized, Role: FOLLOWER
I20260806 18:30:08.163504 18112 consensus_queue.cc:260] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } } peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } }
I20260806 18:30:08.163470 18113 raft_consensus.cc:348] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } } peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } }
I20260806 18:30:08.163754 18113 raft_consensus.cc:374] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:08.163810 18113 raft_consensus.cc:729] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a703950a82f49cb8d102abf747e0f44, State: Initialized, Role: FOLLOWER
I20260806 18:30:08.164279 17701 heartbeater.cc:503] Master 127.7.67.254:39723 was elected leader, sending a full tablet report...
I20260806 18:30:08.164330 18113 consensus_queue.cc:260] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } } peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } }
I20260806 18:30:08.164601 18112 ts_tablet_manager.cc:1428] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260806 18:30:08.165337 18113 ts_tablet_manager.cc:1428] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260806 18:30:08.165694 17885 heartbeater.cc:503] Master 127.7.67.254:39723 was elected leader, sending a full tablet report...
I20260806 18:30:08.167610 18114 raft_consensus.cc:348] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } } peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } }
I20260806 18:30:08.167912 18114 raft_consensus.cc:374] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:08.167986 18114 raft_consensus.cc:729] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09fb11cf02c9434fb56d3d390a1a3536, State: Initialized, Role: FOLLOWER
I20260806 18:30:08.168468 18114 consensus_queue.cc:260] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } } peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } }
I20260806 18:30:08.169677 18114 ts_tablet_manager.cc:1428] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260806 18:30:08.170288 18065 heartbeater.cc:503] Master 127.7.67.254:39723 was elected leader, sending a full tablet report...
I20260806 18:30:08.194779 18123 raft_consensus.cc:482] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:30:08.195061 18123 raft_consensus.cc:504] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } } peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } }
I20260806 18:30:08.196327 18123 leader_election.cc:288] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a703950a82f49cb8d102abf747e0f44 (127.7.67.194:32907), 04c0f654471b4bf19e477ac1f30b535b (127.7.67.193:45973)
I20260806 18:30:08.200098 18109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.196947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:57973 (local address 127.7.67.193:45973)
0806 18:30:08.197069 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:08.197073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:08.197085 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:30:08.197129 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:08.197132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:08.197172 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:08.197234 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:08.197512 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.198014 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.199138 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.199537 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.199644 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.199700 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:08.199878 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 18:30:08.199957 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:30:08.200101 18104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.196676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:56869 (local address 127.7.67.194:32907)
0806 18:30:08.196795 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:08.196798 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:08.196813 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:30:08.196961 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:08.196965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:08.197024 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:08.197100 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:08.197399 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.197988 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.199155 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.199573 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.199681 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.199735 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:08.199901 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 18:30:08.199978 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:30:08.200100 18072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.196596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:45973 (local address 127.7.67.195:57973)
0806 18:30:08.196746 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:08.196757 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:08.196833 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:08.197263 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:08.197266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:08.197287 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:08.197399 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:08.197404 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.198117 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.198121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:08.199005 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:08.199015 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.199652 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.199656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:08.199754 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.199781 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:30:08.199848 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 18:30:08.199960 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":26,"spinlock_wait_cycles":3712}
I20260806 18:30:08.200102 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.196554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:32907 (local address 127.7.67.195:56869)
0806 18:30:08.196717 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:08.196732 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:08.196834 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:08.197132 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:08.197136 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:08.197155 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:08.197279 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:08.197285 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.198100 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.198103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:08.199033 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:08.199044 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.199681 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.199685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:08.199787 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.199808 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:30:08.199880 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 18:30:08.199973 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":63,"spinlock_wait_cycles":8832}
I20260806 18:30:08.200927 17643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa05885955994ae3814408421673529c" candidate_uuid: "09fb11cf02c9434fb56d3d390a1a3536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04c0f654471b4bf19e477ac1f30b535b" is_pre_election: true
I20260806 18:30:08.200937 17820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa05885955994ae3814408421673529c" candidate_uuid: "09fb11cf02c9434fb56d3d390a1a3536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a703950a82f49cb8d102abf747e0f44" is_pre_election: true
I20260806 18:30:08.201233 17820 raft_consensus.cc:2454] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09fb11cf02c9434fb56d3d390a1a3536 in term 0.
I20260806 18:30:08.201229 17643 raft_consensus.cc:2454] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09fb11cf02c9434fb56d3d390a1a3536 in term 0.
I20260806 18:30:08.201705 17917 leader_election.cc:302] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04c0f654471b4bf19e477ac1f30b535b, 09fb11cf02c9434fb56d3d390a1a3536; no voters:
I20260806 18:30:08.201994 18123 raft_consensus.cc:2781] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:30:08.202090 18123 raft_consensus.cc:482] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:30:08.202142 18123 raft_consensus.cc:3037] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:08.203317 18123 raft_consensus.cc:504] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } } peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } }
I20260806 18:30:08.203656 18123 leader_election.cc:288] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [CANDIDATE]: Term 1 election: Requested vote from peers 9a703950a82f49cb8d102abf747e0f44 (127.7.67.194:32907), 04c0f654471b4bf19e477ac1f30b535b (127.7.67.193:45973)
I20260806 18:30:08.204056 17820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa05885955994ae3814408421673529c" candidate_uuid: "09fb11cf02c9434fb56d3d390a1a3536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a703950a82f49cb8d102abf747e0f44"
I20260806 18:30:08.204056 17643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa05885955994ae3814408421673529c" candidate_uuid: "09fb11cf02c9434fb56d3d390a1a3536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04c0f654471b4bf19e477ac1f30b535b"
I20260806 18:30:08.204190 17820 raft_consensus.cc:3037] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:08.204190 17643 raft_consensus.cc:3037] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:08.205471 17820 raft_consensus.cc:2454] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09fb11cf02c9434fb56d3d390a1a3536 in term 1.
I20260806 18:30:08.205471 17643 raft_consensus.cc:2454] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09fb11cf02c9434fb56d3d390a1a3536 in term 1.
I20260806 18:30:08.205776 17914 leader_election.cc:302] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 04c0f654471b4bf19e477ac1f30b535b, 09fb11cf02c9434fb56d3d390a1a3536, 9a703950a82f49cb8d102abf747e0f44; no voters:
I20260806 18:30:08.205963 18123 raft_consensus.cc:2781] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:08.206221 18123 raft_consensus.cc:686] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [term 1 LEADER]: Becoming Leader. State: Replica: 09fb11cf02c9434fb56d3d390a1a3536, State: Running, Role: LEADER
I20260806 18:30:08.206609 18123 consensus_queue.cc:237] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } } peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } }
I20260806 18:30:08.210029 16396 catalog_manager.cc:5495] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09fb11cf02c9434fb56d3d390a1a3536 (127.7.67.195). New cstate: current_term: 1 leader_uuid: "09fb11cf02c9434fb56d3d390a1a3536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } health_report { overall_health: HEALTHY } } }
I20260806 18:30:08.234619 18111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.231092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54834 (local address 127.7.67.195:45549)
0806 18:30:08.231225 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:08.231229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:08.231243 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:30:08.231319 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:08.231322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:08.231377 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:08.231468 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:30:08.231743 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.232271 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.233117 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.233329 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.233425 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.233744 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:30:08.234208 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:30:08.234316 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:08.234390 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 18:30:08.234452 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:30:08.240366 17820 raft_consensus.cc:1261] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44 [term 1 FOLLOWER]: Refusing update from remote peer 09fb11cf02c9434fb56d3d390a1a3536: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:30:08.240360 17643 raft_consensus.cc:1261] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b [term 1 FOLLOWER]: Refusing update from remote peer 09fb11cf02c9434fb56d3d390a1a3536: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:30:08.241078 18124 consensus_queue.cc:1035] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:30:08.241262 18123 consensus_queue.cc:1035] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:30:08.250885 18128 mvcc.cc:205] Tried to move back new op lower bound from 7315623969745399808 to 7315623969617362944. Current Snapshot: MvccSnapshot[applied={T|T < 7315623969745399808}]
I20260806 18:30:08.253031 18129 mvcc.cc:205] Tried to move back new op lower bound from 7315623969745399808 to 7315623969617362944. Current Snapshot: MvccSnapshot[applied={T|T < 7315623969745399808}]
I20260806 18:30:08.260905 16396 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:30:08.261062 16396 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:30:08.265264 16396 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c6a620695abe4ec7b4577e405a25f199: Sending DeleteTablet for 3 replicas of tablet aa05885955994ae3814408421673529c
I20260806 18:30:08.266223 17611 tablet_service.cc:1514] Processing DeleteTablet for tablet aa05885955994ae3814408421673529c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:08 UTC) from {username='oryx', principal='oryx/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:48616
I20260806 18:30:08.266255 17795 tablet_service.cc:1514] Processing DeleteTablet for tablet aa05885955994ae3814408421673529c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:08 UTC) from {username='oryx', principal='oryx/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:37986
I20260806 18:30:08.266794 17978 tablet_service.cc:1514] Processing DeleteTablet for tablet aa05885955994ae3814408421673529c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:08 UTC) from {username='oryx', principal='oryx/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:54826
I20260806 18:30:08.267328 18145 tablet_replica.cc:330] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536: stopping tablet replica
I20260806 18:30:08.267405 18139 tablet_replica.cc:330] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b: stopping tablet replica
I20260806 18:30:08.267766 18139 raft_consensus.cc:2229] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:30:08.267803 18145 raft_consensus.cc:2229] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:30:08.268131 18139 raft_consensus.cc:2258] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:08.268378 18145 raft_consensus.cc:2258] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:08.269850 18139 ts_tablet_manager.cc:1905] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:08.270051 18145 ts_tablet_manager.cc:1905] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:08.270200 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 16350
I20260806 18:30:08.280030 18139 ts_tablet_manager.cc:1918] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:30:08.280169 18139 log.cc:1196] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-0/wal/wals/aa05885955994ae3814408421673529c
I20260806 18:30:08.280480 18139 ts_tablet_manager.cc:1939] T aa05885955994ae3814408421673529c P 04c0f654471b4bf19e477ac1f30b535b: Deleting consensus metadata
I20260806 18:30:08.282066 18142 tablet_replica.cc:330] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44: stopping tablet replica
I20260806 18:30:08.282480 18142 raft_consensus.cc:2229] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:30:08.282841 18142 raft_consensus.cc:2258] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:08.282989 18145 ts_tablet_manager.cc:1918] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:30:08.284727 18142 ts_tablet_manager.cc:1905] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:08.284920 18145 log.cc:1196] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/aa05885955994ae3814408421673529c
I20260806 18:30:08.285332 18145 ts_tablet_manager.cc:1939] T aa05885955994ae3814408421673529c P 09fb11cf02c9434fb56d3d390a1a3536: Deleting consensus metadata
I20260806 18:30:08.288137 18142 ts_tablet_manager.cc:1918] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260806 18:30:08.289489 17546 connection.cc:535] server connection from 127.0.0.1:48616 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 18:30:08.289662 18142 log.cc:1196] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/aa05885955994ae3814408421673529c
W20260806 18:30:08.289876 17914 connection.cc:441] server connection from 127.0.0.1:54826 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54826 (request call id 1) could send its response
I20260806 18:30:08.290233 18142 ts_tablet_manager.cc:1939] T aa05885955994ae3814408421673529c P 9a703950a82f49cb8d102abf747e0f44: Deleting consensus metadata
W20260806 18:30:08.291620 17736 connection.cc:441] server connection from 127.0.0.1:37986 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37986 (request call id 1) could send its response
I20260806 18:30:08.294659 18081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.269646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41626 (local address 127.7.67.253:38821)
0806 18:30:08.269768 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:08.269772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:08.283734 (+ 13962us) server_negotiation.cc:407] Connection header received
0806 18:30:08.284127 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:08.284132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:08.284201 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:08.284255 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:08.285400 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.286195 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.287435 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.287778 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.287865 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.289971 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:08.289975 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:08.289977 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:08.290008 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:08.292011 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.292517 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.292521 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.292523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.292628 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.292975 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.292979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.292980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.293251 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:08.293398 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:08.294354 (+ 956us) server_negotiation.cc:299] Negotiation successful
0806 18:30:08.294490 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 18:30:08.303406 17370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.294541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41636 (local address 127.7.67.253:38821)
0806 18:30:08.294662 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:08.294666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:08.294968 (+ 302us) server_negotiation.cc:407] Connection header received
0806 18:30:08.295135 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:08.295138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:08.295182 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:08.295231 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:08.296114 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.296846 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.297908 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.298211 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.298288 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.299852 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:08.299856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:08.299859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:08.299884 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:08.301507 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.302032 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.302036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.302037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.302128 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.302430 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.302433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.302435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.302651 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:08.302770 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:08.303134 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 18:30:08.303242 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:30:08.303658 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.269437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40228 (local address 127.7.67.252:40865)
0806 18:30:08.269556 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:08.269561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:08.294104 (+ 24543us) server_negotiation.cc:407] Connection header received
0806 18:30:08.294174 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:08.294177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:08.294237 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:08.294322 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:08.295175 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.295970 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.297298 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.297653 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.297741 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.299680 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:08.299684 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:08.299687 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:08.299714 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:08.301660 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.302109 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.302112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.302114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.302196 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.302463 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.302466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.302468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.302651 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:08.302769 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:08.303380 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 18:30:08.303514 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:30:08.306110 17895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:08.296153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40236 (local address 127.7.67.252:40865)
0806 18:30:08.296290 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:08.296294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:08.296545 (+ 251us) server_negotiation.cc:407] Connection header received
0806 18:30:08.296701 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:08.296704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:08.296746 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:08.296792 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:08.297704 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.298493 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.299754 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:08.300120 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:08.300191 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:08.301858 (+ 1667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:08.301862 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:08.301864 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:08.301891 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:08.304565 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.305020 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.305023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.305024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.305101 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:08.305381 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:08.305384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:08.305385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:08.305565 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:08.305665 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:08.305878 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 18:30:08.306001 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:30:09.253739 18158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:09.253214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.195:56723)
0806 18:30:09.253485 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:09.253597 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":81,"threads_started":1}
W20260806 18:30:09.254156 18065 heartbeater.cc:644] Failed to heartbeat to 127.7.67.254:39723 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
I20260806 18:30:09.254557 18159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:09.254028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.193:41469)
0806 18:30:09.254356 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:09.254451 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":76,"threads_started":1}
W20260806 18:30:09.254825 17701 heartbeater.cc:644] Failed to heartbeat to 127.7.67.254:39723 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
I20260806 18:30:09.255957 18163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:09.255569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.194:60037)
0806 18:30:09.255780 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:09.255860 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":60,"threads_started":1}
W20260806 18:30:09.256356 17885 heartbeater.cc:644] Failed to heartbeat to 127.7.67.254:39723 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
I20260806 18:30:09.766104 18164 raft_consensus.cc:482] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c6a620695abe4ec7b4577e405a25f199)
I20260806 18:30:09.766338 18164 raft_consensus.cc:504] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } }
I20260806 18:30:09.766835 18165 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c6a620695abe4ec7b4577e405a25f199)
I20260806 18:30:09.767050 18165 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } }
I20260806 18:30:09.767400 18164 leader_election.cc:288] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c6a620695abe4ec7b4577e405a25f199 (127.7.67.254:39723), 7f88490fdc154ddbbbe5a0f3a68c2227 (127.7.67.252:40865)
I20260806 18:30:09.768014 18165 leader_election.cc:288] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c6a620695abe4ec7b4577e405a25f199 (127.7.67.254:39723), bec8fce3919c48a4b5d7e5cfb2a96b11 (127.7.67.253:38821)
I20260806 18:30:09.768535 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:09.768074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54522)
0806 18:30:09.768340 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:09.768414 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":71,"threads_started":1}
I20260806 18:30:09.768535 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:09.767549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54510)
0806 18:30:09.767822 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:09.767907 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":57,"threads_started":1}
W20260806 18:30:09.768736 17195 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
W20260806 18:30:09.768718 17433 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
I20260806 18:30:09.769590 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:09.769388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54534)
0806 18:30:09.769485 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:09.769533 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 18:30:09.769721 17433 leader_election.cc:334] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c6a620695abe4ec7b4577e405a25f199 (127.7.67.254:39723): Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
I20260806 18:30:09.770309 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:09.770056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54544)
0806 18:30:09.770185 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:09.770239 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 18:30:09.770428 17195 leader_election.cc:334] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c6a620695abe4ec7b4577e405a25f199 (127.7.67.254:39723): Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
I20260806 18:30:09.772133 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:09.767622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:40865 (local address 127.0.0.1:40250)
0806 18:30:09.767967 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:09.767981 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:09.768163 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:09.768699 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:09.768704 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:09.768726 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:09.768950 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:09.768959 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:09.769997 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:09.770001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:09.771029 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:09.771038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:09.771696 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:09.771700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:09.771845 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:09.771872 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:30:09.771941 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 18:30:09.772024 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":33,"spinlock_wait_cycles":4352,"thread_start_us":70,"threads_started":1}
I20260806 18:30:09.772156 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:09.767778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40250 (local address 127.7.67.252:40865)
0806 18:30:09.767969 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:09.767974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:09.768144 (+ 170us) server_negotiation.cc:407] Connection header received
0806 18:30:09.768309 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:09.768313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:09.768365 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:09.768438 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:09.769070 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:09.769874 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:09.771143 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:09.771594 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:09.771671 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:09.771720 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:09.771954 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 18:30:09.772027 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":52,"threads_started":1}
I20260806 18:30:09.772297 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:09.768165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:38821 (local address 127.0.0.1:41638)
0806 18:30:09.768425 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:09.768438 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:09.768500 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:09.768917 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:09.768920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:09.768937 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:09.769044 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:09.769051 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:09.770202 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:09.770205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:09.771217 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:09.771227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:09.771935 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:09.771939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:09.772032 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:09.772053 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:30:09.772127 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:30:09.772191 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":69,"threads_started":1}
I20260806 18:30:09.772631 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:09.768318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41638 (local address 127.7.67.253:38821)
0806 18:30:09.768693 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:09.768698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:09.768709 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:30:09.768745 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:09.768747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:09.768794 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:09.769136 (+ 342us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:09.769223 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:09.770082 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:09.771335 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:09.771828 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:09.771924 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:09.771976 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:09.772150 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 18:30:09.772533 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":314,"spinlock_wait_cycles":3200,"thread_start_us":59,"threads_started":1}
I20260806 18:30:09.772605 17466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" is_pre_election: true
I20260806 18:30:09.772794 17466 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bec8fce3919c48a4b5d7e5cfb2a96b11 in term 1.
I20260806 18:30:09.772982 17224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" is_pre_election: true
I20260806 18:30:09.773154 17192 leader_election.cc:302] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f88490fdc154ddbbbe5a0f3a68c2227, bec8fce3919c48a4b5d7e5cfb2a96b11; no voters: c6a620695abe4ec7b4577e405a25f199
I20260806 18:30:09.773177 17224 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f88490fdc154ddbbbe5a0f3a68c2227 in term 1.
I20260806 18:30:09.773376 18164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 18:30:09.773432 18164 raft_consensus.cc:482] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c6a620695abe4ec7b4577e405a25f199)
I20260806 18:30:09.773469 18164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:30:09.773537 17432 leader_election.cc:302] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f88490fdc154ddbbbe5a0f3a68c2227, bec8fce3919c48a4b5d7e5cfb2a96b11; no voters: c6a620695abe4ec7b4577e405a25f199
I20260806 18:30:09.773715 18165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 18:30:09.773764 18165 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c6a620695abe4ec7b4577e405a25f199)
I20260806 18:30:09.773802 18165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:30:09.774475 18164 raft_consensus.cc:504] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } }
I20260806 18:30:09.774737 18165 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } }
I20260806 18:30:09.774794 18164 leader_election.cc:288] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [CANDIDATE]: Term 2 election: Requested vote from peers c6a620695abe4ec7b4577e405a25f199 (127.7.67.254:39723), 7f88490fdc154ddbbbe5a0f3a68c2227 (127.7.67.252:40865)
I20260806 18:30:09.775112 18165 leader_election.cc:288] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [CANDIDATE]: Term 2 election: Requested vote from peers c6a620695abe4ec7b4577e405a25f199 (127.7.67.254:39723), bec8fce3919c48a4b5d7e5cfb2a96b11 (127.7.67.253:38821)
I20260806 18:30:09.775151 17466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227"
I20260806 18:30:09.775282 17466 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bec8fce3919c48a4b5d7e5cfb2a96b11 in current term 2: Already voted for candidate 7f88490fdc154ddbbbe5a0f3a68c2227 in this term.
I20260806 18:30:09.775398 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:09.775196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54564)
0806 18:30:09.775289 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:09.775336 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 18:30:09.775202 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:09.774942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54556)
0806 18:30:09.775090 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:09.775140 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 18:30:09.775681 17224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11"
I20260806 18:30:09.775835 17224 raft_consensus.cc:2379] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7f88490fdc154ddbbbe5a0f3a68c2227 in current term 2: Already voted for candidate bec8fce3919c48a4b5d7e5cfb2a96b11 in this term.
I20260806 18:30:09.776332 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:09.775764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54580)
0806 18:30:09.776017 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:09.776269 (+ 252us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180}
I20260806 18:30:09.776337 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:09.776146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54588)
0806 18:30:09.776242 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:09.776283 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 18:30:09.776444 17195 leader_election.cc:334] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c6a620695abe4ec7b4577e405a25f199 (127.7.67.254:39723): Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
I20260806 18:30:09.776518 17195 leader_election.cc:302] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bec8fce3919c48a4b5d7e5cfb2a96b11; no voters: 7f88490fdc154ddbbbe5a0f3a68c2227, c6a620695abe4ec7b4577e405a25f199
W20260806 18:30:09.776516 17433 leader_election.cc:334] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c6a620695abe4ec7b4577e405a25f199 (127.7.67.254:39723): Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
I20260806 18:30:09.776571 17433 leader_election.cc:302] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f88490fdc154ddbbbe5a0f3a68c2227; no voters: bec8fce3919c48a4b5d7e5cfb2a96b11, c6a620695abe4ec7b4577e405a25f199
I20260806 18:30:09.776696 18164 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 18:30:09.776702 18165 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 18:30:10.255522 18176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:10.254991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.195:58411)
0806 18:30:10.255316 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:10.255410 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":106,"threads_started":1}
I20260806 18:30:10.255940 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:10.255435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.193:53775)
0806 18:30:10.255772 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:10.255837 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":89,"threads_started":1}
I20260806 18:30:10.257323 18178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:10.256874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.194:40517)
0806 18:30:10.257163 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:10.257227 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":98,"threads_started":1}
I20260806 18:30:11.257056 18181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.256633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.193:40313)
0806 18:30:11.256918 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:11.256973 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":79,"threads_started":1}
I20260806 18:30:11.257036 18180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.256546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.195:40921)
0806 18:30:11.256836 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:11.256912 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":90,"threads_started":1}
I20260806 18:30:11.258474 18182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.258002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.194:43613)
0806 18:30:11.258289 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:11.258373 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":68,"threads_started":1}
I20260806 18:30:11.825031 18183 raft_consensus.cc:482] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:30:11.825215 18183 raft_consensus.cc:504] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } }
I20260806 18:30:11.825685 18183 leader_election.cc:288] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c6a620695abe4ec7b4577e405a25f199 (127.7.67.254:39723), 7f88490fdc154ddbbbe5a0f3a68c2227 (127.7.67.252:40865)
I20260806 18:30:11.826174 17466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" is_pre_election: true
I20260806 18:30:11.826339 17466 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bec8fce3919c48a4b5d7e5cfb2a96b11 in term 2.
I20260806 18:30:11.826323 18185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.825922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54858)
0806 18:30:11.826166 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:11.826228 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":85,"threads_started":1}
I20260806 18:30:11.826684 17192 leader_election.cc:302] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f88490fdc154ddbbbe5a0f3a68c2227, bec8fce3919c48a4b5d7e5cfb2a96b11; no voters:
I20260806 18:30:11.826875 18183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 18:30:11.826936 18183 raft_consensus.cc:482] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:30:11.826974 18183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 2 FOLLOWER]: Advancing to term 3
I20260806 18:30:11.827245 18185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.827017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54872)
0806 18:30:11.827122 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:11.827178 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 18:30:11.827387 17195 leader_election.cc:334] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c6a620695abe4ec7b4577e405a25f199 (127.7.67.254:39723): Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
I20260806 18:30:11.828132 18183 raft_consensus.cc:504] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } }
I20260806 18:30:11.828450 18183 leader_election.cc:288] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [CANDIDATE]: Term 3 election: Requested vote from peers c6a620695abe4ec7b4577e405a25f199 (127.7.67.254:39723), 7f88490fdc154ddbbbe5a0f3a68c2227 (127.7.67.252:40865)
I20260806 18:30:11.828845 18185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.828604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54882)
0806 18:30:11.828709 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:11.828764 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 18:30:11.828855 17466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227"
I20260806 18:30:11.829000 17466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 2 FOLLOWER]: Advancing to term 3
I20260806 18:30:11.829453 18185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.829255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54894)
0806 18:30:11.829352 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:11.829398 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 18:30:11.829556 17195 leader_election.cc:334] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer c6a620695abe4ec7b4577e405a25f199 (127.7.67.254:39723): Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
I20260806 18:30:11.830132 17466 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bec8fce3919c48a4b5d7e5cfb2a96b11 in term 3.
I20260806 18:30:11.830430 17192 leader_election.cc:302] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f88490fdc154ddbbbe5a0f3a68c2227, bec8fce3919c48a4b5d7e5cfb2a96b11; no voters: c6a620695abe4ec7b4577e405a25f199
I20260806 18:30:11.830595 18183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 18:30:11.830770 18183 raft_consensus.cc:686] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [term 3 LEADER]: Becoming Leader. State: Replica: bec8fce3919c48a4b5d7e5cfb2a96b11, State: Running, Role: LEADER
I20260806 18:30:11.831051 18183 consensus_queue.cc:237] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } }
I20260806 18:30:11.832098 18187 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bec8fce3919c48a4b5d7e5cfb2a96b11. Latest consensus state: current_term: 3 leader_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } } }
I20260806 18:30:11.832224 18187 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:11.832535 18189 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:30:11.834997 18189 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:30:11.835457 18189 catalog_manager.cc:1259] Loaded cluster ID: 046fb588ad9f453898a7561cc299683e
I20260806 18:30:11.835502 18189 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:30:11.836076 18189 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:30:11.836447 18189 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: Loaded TSK: 0
I20260806 18:30:11.837116 18189 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:30:11.845993 17210 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:30:11.849306 17210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:30:11.852407 18185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.852168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54918)
0806 18:30:11.852287 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:11.852339 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 18:30:11.852442 17466 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [term 3 FOLLOWER]: Refusing update from remote peer bec8fce3919c48a4b5d7e5cfb2a96b11: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 18:30:11.852938 18187 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 18:30:11.853125 18185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.852906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.0.0.1:54930)
0806 18:30:11.853006 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:11.853057 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 18:30:11.853268 17195 consensus_peers.cc:487] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 -> Peer c6a620695abe4ec7b4577e405a25f199 (127.7.67.254:39723): Couldn't send request to peer c6a620695abe4ec7b4577e405a25f199. Status: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 18:30:11.854445 18191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bec8fce3919c48a4b5d7e5cfb2a96b11. Latest consensus state: current_term: 3 leader_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } } }
I20260806 18:30:11.854611 18191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:30:11.855611 18183 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } } }
I20260806 18:30:11.855727 18183 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:11.856106 18191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6a620695abe4ec7b4577e405a25f199" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 39723 } } peers { permanent_uuid: "bec8fce3919c48a4b5d7e5cfb2a96b11" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 38821 } attrs { promote: false } } peers { permanent_uuid: "7f88490fdc154ddbbbe5a0f3a68c2227" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 40865 } attrs { promote: false } } }
I20260806 18:30:11.856222 18191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f88490fdc154ddbbbe5a0f3a68c2227 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:30:11.856564 18192 mvcc.cc:205] Tried to move back new op lower bound from 7315623984542699520 to 7315623984462356480. Current Snapshot: MvccSnapshot[applied={T|T < 7315623984542699520}]
I20260806 18:30:11.865535 18185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.861168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:45549 (local address 127.0.0.1:54838)
0806 18:30:11.861285 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:11.861299 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:11.861406 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:11.861908 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:11.861911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:11.861925 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:11.862021 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:11.862026 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:11.862898 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:11.862902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:11.863650 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:11.863658 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:11.865175 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:11.865177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:11.865252 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:11.865274 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:30:11.865330 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 18:30:11.865398 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 18:30:11.865592 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.861304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54838 (local address 127.7.67.195:45549)
0806 18:30:11.861692 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:11.861697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:11.861710 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:30:11.861761 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:11.861764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:11.861811 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:11.861887 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:11.862150 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:11.862746 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:11.864409 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:11.865063 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:11.865162 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:11.865230 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:11.865361 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 18:30:11.865447 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":83,"threads_started":1}
I20260806 18:30:11.866083 17978 tablet_service.cc:1467] Processing CreateTablet for tablet f53c958d83934f4dbbbdb8691f5b2969 (DEFAULT_TABLE table=test-table [id=b2ab67e55f524e0eb3aa0629457fd7a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:11.866386 17978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53c958d83934f4dbbbdb8691f5b2969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:11.866780 18207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.861397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:45973 (local address 127.0.0.1:48624)
0806 18:30:11.861641 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:11.861655 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:11.861745 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:11.862649 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:11.862652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:11.862670 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:11.862766 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:11.862773 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:11.863764 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:11.863768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:11.865702 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:11.865713 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:11.866398 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:11.866402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:11.866493 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:11.866511 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:30:11.866580 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 18:30:11.866649 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":1049,"thread_start_us":100,"threads_started":1}
I20260806 18:30:11.866797 18208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.861794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48624 (local address 127.7.67.193:45973)
0806 18:30:11.862422 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:11.862427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:11.862443 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:30:11.862496 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:11.862500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:11.862545 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:11.862624 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:11.862924 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:11.863655 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:11.865834 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:11.866286 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:11.866381 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:11.866443 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:11.866607 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 18:30:11.866682 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":91,"threads_started":1}
I20260806 18:30:11.867290 17611 tablet_service.cc:1467] Processing CreateTablet for tablet f53c958d83934f4dbbbdb8691f5b2969 (DEFAULT_TABLE table=test-table [id=b2ab67e55f524e0eb3aa0629457fd7a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:11.867529 17611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53c958d83934f4dbbbdb8691f5b2969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:11.868940 18213 tablet_bootstrap.cc:492] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536: Bootstrap starting.
I20260806 18:30:11.869837 18214 tablet_bootstrap.cc:492] T f53c958d83934f4dbbbdb8691f5b2969 P 04c0f654471b4bf19e477ac1f30b535b: Bootstrap starting.
I20260806 18:30:11.869910 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.862109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:32907 (local address 127.0.0.1:37988)
0806 18:30:11.862816 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:11.862829 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:11.862956 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:11.863312 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:11.863315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:11.863331 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:11.863416 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:11.863422 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:11.864216 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:11.864220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:11.865298 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:11.865309 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:11.869507 (+ 4198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:11.869511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:11.869606 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:11.869631 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:30:11.869699 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 18:30:11.869772 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"mutex_wait_us":1,"thread_start_us":84,"threads_started":1}
I20260806 18:30:11.870007 18213 tablet_bootstrap.cc:654] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:11.870437 18212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.862229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37988 (local address 127.7.67.194:32907)
0806 18:30:11.862465 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:11.862470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:11.863032 (+ 562us) server_negotiation.cc:407] Connection header received
0806 18:30:11.863089 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:11.863092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:11.863138 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:11.863210 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:11.863533 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:11.864118 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:11.865429 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:11.869247 (+ 3818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:11.869358 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:11.869439 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:11.870192 (+ 753us) server_negotiation.cc:299] Negotiation successful
0806 18:30:11.870291 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":65,"threads_started":1}
I20260806 18:30:11.870793 18214 tablet_bootstrap.cc:654] T f53c958d83934f4dbbbdb8691f5b2969 P 04c0f654471b4bf19e477ac1f30b535b: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:11.870981 17795 tablet_service.cc:1467] Processing CreateTablet for tablet f53c958d83934f4dbbbdb8691f5b2969 (DEFAULT_TABLE table=test-table [id=b2ab67e55f524e0eb3aa0629457fd7a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:11.871256 17795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53c958d83934f4dbbbdb8691f5b2969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:11.871349 18213 tablet_bootstrap.cc:492] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536: No bootstrap required, opened a new log
I20260806 18:30:11.871446 18213 ts_tablet_manager.cc:1397] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:30:11.871874 18213 raft_consensus.cc:348] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } } peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } }
I20260806 18:30:11.871992 18213 raft_consensus.cc:374] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:11.872027 18213 raft_consensus.cc:729] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09fb11cf02c9434fb56d3d390a1a3536, State: Initialized, Role: FOLLOWER
I20260806 18:30:11.872159 18214 tablet_bootstrap.cc:492] T f53c958d83934f4dbbbdb8691f5b2969 P 04c0f654471b4bf19e477ac1f30b535b: No bootstrap required, opened a new log
I20260806 18:30:11.872150 18213 consensus_queue.cc:260] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } } peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } }
I20260806 18:30:11.872233 18214 ts_tablet_manager.cc:1397] T f53c958d83934f4dbbbdb8691f5b2969 P 04c0f654471b4bf19e477ac1f30b535b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:30:11.872498 18213 ts_tablet_manager.cc:1428] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:30:11.872653 18214 raft_consensus.cc:348] T f53c958d83934f4dbbbdb8691f5b2969 P 04c0f654471b4bf19e477ac1f30b535b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } } peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } }
I20260806 18:30:11.872753 18214 raft_consensus.cc:374] T f53c958d83934f4dbbbdb8691f5b2969 P 04c0f654471b4bf19e477ac1f30b535b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:11.872784 18214 raft_consensus.cc:729] T f53c958d83934f4dbbbdb8691f5b2969 P 04c0f654471b4bf19e477ac1f30b535b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04c0f654471b4bf19e477ac1f30b535b, State: Initialized, Role: FOLLOWER
I20260806 18:30:11.872900 18214 consensus_queue.cc:260] T f53c958d83934f4dbbbdb8691f5b2969 P 04c0f654471b4bf19e477ac1f30b535b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } } peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } }
I20260806 18:30:11.873206 18214 ts_tablet_manager.cc:1428] T f53c958d83934f4dbbbdb8691f5b2969 P 04c0f654471b4bf19e477ac1f30b535b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:30:11.874270 18223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.873865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.193:38933)
0806 18:30:11.874117 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:11.874179 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":96,"threads_started":1}
I20260806 18:30:11.875133 18223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.874703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.193:59389)
0806 18:30:11.875007 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:11.875061 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":85,"threads_started":1}
I20260806 18:30:11.875283 18219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.872755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.195:58305)
0806 18:30:11.875146 (+ 2391us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:11.875199 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2327,"thread_start_us":94,"threads_started":1}
I20260806 18:30:11.875587 18221 tablet_bootstrap.cc:492] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44: Bootstrap starting.
I20260806 18:30:11.876112 17703 heartbeater.cc:503] Master 127.7.67.253:38821 was elected leader, sending a full tablet report...
I20260806 18:30:11.876364 18068 heartbeater.cc:503] Master 127.7.67.253:38821 was elected leader, sending a full tablet report...
I20260806 18:30:11.876933 18221 tablet_bootstrap.cc:654] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:11.878106 18219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.875695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.195:57535)
0806 18:30:11.877985 (+ 2290us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:11.878040 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2218}
I20260806 18:30:11.878171 18221 tablet_bootstrap.cc:492] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44: No bootstrap required, opened a new log
I20260806 18:30:11.878259 18221 ts_tablet_manager.cc:1397] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:30:11.878652 18221 raft_consensus.cc:348] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } } peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } }
I20260806 18:30:11.878767 18221 raft_consensus.cc:374] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:11.878798 18221 raft_consensus.cc:729] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a703950a82f49cb8d102abf747e0f44, State: Initialized, Role: FOLLOWER
I20260806 18:30:11.878947 18221 consensus_queue.cc:260] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } } peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } }
I20260806 18:30:11.879274 18221 ts_tablet_manager.cc:1428] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:30:11.880218 18228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.879691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.194:46671)
0806 18:30:11.879980 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:11.880065 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":104,"threads_started":1}
I20260806 18:30:11.882159 17888 heartbeater.cc:503] Master 127.7.67.253:38821 was elected leader, sending a full tablet report...
I20260806 18:30:11.882902 18228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:11.882655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.194:45653)
0806 18:30:11.882757 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:11.882812 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 18:30:11.913069 17704 tablet.cc:2369] T f53c958d83934f4dbbbdb8691f5b2969 P 04c0f654471b4bf19e477ac1f30b535b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:30:12.087718 18069 tablet.cc:2369] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:30:12.103713 17890 tablet.cc:2369] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:30:12.135200 18227 raft_consensus.cc:482] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:30:12.135358 18227 raft_consensus.cc:504] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } } peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } }
I20260806 18:30:12.136478 18227 leader_election.cc:288] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09fb11cf02c9434fb56d3d390a1a3536 (127.7.67.195:45549), 04c0f654471b4bf19e477ac1f30b535b (127.7.67.193:45973)
I20260806 18:30:12.140211 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:12.136891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:36339 (local address 127.7.67.195:45549)
0806 18:30:12.137054 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:12.137059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:12.137078 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:30:12.137168 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:12.137171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:12.137229 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:12.137307 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:12.137656 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:12.138161 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:12.139252 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:12.139652 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:12.139756 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:12.139813 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:12.140016 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 18:30:12.140086 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:30:12.140221 18228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:12.136757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:45549 (local address 127.7.67.194:36339)
0806 18:30:12.136923 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:12.136940 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:12.137073 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:12.137416 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:12.137419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:12.137437 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:12.137540 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:12.137544 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:12.138269 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:12.138272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:12.139131 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:12.139140 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:12.139795 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:12.139799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:12.139897 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:12.139916 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:30:12.139991 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 18:30:12.140062 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":1}
I20260806 18:30:12.140266 18208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:12.137171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:47707 (local address 127.7.67.193:45973)
0806 18:30:12.137293 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:12.137298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:12.137437 (+ 139us) server_negotiation.cc:407] Connection header received
0806 18:30:12.137542 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:12.137545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:12.137591 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:12.137652 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:12.137893 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:12.138385 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:12.139377 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:12.139755 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:12.139858 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:12.139907 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:12.140079 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 18:30:12.140146 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:30:12.140319 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:12.137010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:45973 (local address 127.7.67.194:47707)
0806 18:30:12.137355 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:12.137365 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:12.137456 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:12.137676 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:12.137678 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:12.137694 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:12.137793 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:12.137798 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:12.138487 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:12.138491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:12.139257 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:12.139266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:12.139863 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:12.139867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:12.139962 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:12.139982 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:30:12.140044 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 18:30:12.140113 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":136,"threads_started":1}
I20260806 18:30:12.140712 17643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53c958d83934f4dbbbdb8691f5b2969" candidate_uuid: "9a703950a82f49cb8d102abf747e0f44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04c0f654471b4bf19e477ac1f30b535b" is_pre_election: true
I20260806 18:30:12.140770 18000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53c958d83934f4dbbbdb8691f5b2969" candidate_uuid: "9a703950a82f49cb8d102abf747e0f44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09fb11cf02c9434fb56d3d390a1a3536" is_pre_election: true
I20260806 18:30:12.140869 17643 raft_consensus.cc:2454] T f53c958d83934f4dbbbdb8691f5b2969 P 04c0f654471b4bf19e477ac1f30b535b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a703950a82f49cb8d102abf747e0f44 in term 0.
I20260806 18:30:12.141002 18000 raft_consensus.cc:2454] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a703950a82f49cb8d102abf747e0f44 in term 0.
I20260806 18:30:12.141273 17735 leader_election.cc:302] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04c0f654471b4bf19e477ac1f30b535b, 9a703950a82f49cb8d102abf747e0f44; no voters:
I20260806 18:30:12.141512 18227 raft_consensus.cc:2781] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:30:12.141570 18227 raft_consensus.cc:482] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:30:12.141594 18227 raft_consensus.cc:3037] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:12.144739 18227 raft_consensus.cc:504] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } } peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } }
I20260806 18:30:12.145169 18227 leader_election.cc:288] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [CANDIDATE]: Term 1 election: Requested vote from peers 09fb11cf02c9434fb56d3d390a1a3536 (127.7.67.195:45549), 04c0f654471b4bf19e477ac1f30b535b (127.7.67.193:45973)
I20260806 18:30:12.145555 18000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53c958d83934f4dbbbdb8691f5b2969" candidate_uuid: "9a703950a82f49cb8d102abf747e0f44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09fb11cf02c9434fb56d3d390a1a3536"
I20260806 18:30:12.145555 17643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53c958d83934f4dbbbdb8691f5b2969" candidate_uuid: "9a703950a82f49cb8d102abf747e0f44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04c0f654471b4bf19e477ac1f30b535b"
I20260806 18:30:12.145685 17643 raft_consensus.cc:3037] T f53c958d83934f4dbbbdb8691f5b2969 P 04c0f654471b4bf19e477ac1f30b535b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:12.145685 18000 raft_consensus.cc:3037] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:12.146934 18000 raft_consensus.cc:2454] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a703950a82f49cb8d102abf747e0f44 in term 1.
I20260806 18:30:12.146937 17643 raft_consensus.cc:2454] T f53c958d83934f4dbbbdb8691f5b2969 P 04c0f654471b4bf19e477ac1f30b535b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a703950a82f49cb8d102abf747e0f44 in term 1.
I20260806 18:30:12.147238 17735 leader_election.cc:302] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09fb11cf02c9434fb56d3d390a1a3536, 9a703950a82f49cb8d102abf747e0f44; no voters:
I20260806 18:30:12.147396 18227 raft_consensus.cc:2781] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:12.147670 18227 raft_consensus.cc:686] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [term 1 LEADER]: Becoming Leader. State: Replica: 9a703950a82f49cb8d102abf747e0f44, State: Running, Role: LEADER
I20260806 18:30:12.147933 18227 consensus_queue.cc:237] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } } peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } }
I20260806 18:30:12.148325 18228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:12.148081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.194:43139)
0806 18:30:12.148185 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:12.148249 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 18:30:12.150493 17210 catalog_manager.cc:5495] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a703950a82f49cb8d102abf747e0f44 (127.7.67.194). New cstate: current_term: 1 leader_uuid: "9a703950a82f49cb8d102abf747e0f44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a703950a82f49cb8d102abf747e0f44" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 32907 } health_report { overall_health: HEALTHY } } }
I20260806 18:30:12.230064 18212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:12.226539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38004 (local address 127.7.67.194:32907)
0806 18:30:12.226674 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:12.226679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:12.226693 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:30:12.226891 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:12.226894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:12.226956 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:12.227049 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:30:12.227295 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:12.227803 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:12.228656 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:12.228877 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:12.228963 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:12.229225 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:30:12.229682 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:30:12.229785 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:12.229860 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 18:30:12.229920 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:30:12.233137 17643 raft_consensus.cc:1261] T f53c958d83934f4dbbbdb8691f5b2969 P 04c0f654471b4bf19e477ac1f30b535b [term 1 FOLLOWER]: Refusing update from remote peer 9a703950a82f49cb8d102abf747e0f44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:30:12.233341 18000 raft_consensus.cc:1261] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536 [term 1 FOLLOWER]: Refusing update from remote peer 9a703950a82f49cb8d102abf747e0f44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:30:12.233783 18230 consensus_queue.cc:1035] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04c0f654471b4bf19e477ac1f30b535b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:30:12.234059 18227 consensus_queue.cc:1035] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09fb11cf02c9434fb56d3d390a1a3536" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:30:12.235915 18219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:12.235616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.195:43017)
0806 18:30:12.235766 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:12.235822 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 18:30:12.235918 18225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:12.235618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.193:43689)
0806 18:30:12.235745 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:12.235810 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 18:30:12.239253 18228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:12.239010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.194:59029)
0806 18:30:12.239110 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:12.239169 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 18:30:12.240927 18228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:12.240497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.194:56333)
0806 18:30:12.240799 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:12.240853 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233}
I20260806 18:30:12.240975 18219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:12.240728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.195:59167)
0806 18:30:12.240860 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:12.240916 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 18:30:12.241973 18225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:12.241659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:39723 (local address 127.7.67.193:58269)
0806 18:30:12.241846 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:12.241900 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260806 18:30:12.245282 18237 mvcc.cc:205] Tried to move back new op lower bound from 7315623986101047296 to 7315623985760763904. Current Snapshot: MvccSnapshot[applied={T|T < 7315623986101047296 or (T in {7315623986101047296})}]
I20260806 18:30:12.247414 17210 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:30:12.247592 17210 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:30:12.251830 17210 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bec8fce3919c48a4b5d7e5cfb2a96b11: Sending DeleteTablet for 3 replicas of tablet f53c958d83934f4dbbbdb8691f5b2969
I20260806 18:30:12.252533 17978 tablet_service.cc:1514] Processing DeleteTablet for tablet f53c958d83934f4dbbbdb8691f5b2969 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:12 UTC) from {username='oryx', principal='oryx/127.7.67.253@KRBTEST.COM'} at 127.0.0.1:54838
I20260806 18:30:12.252866 17795 tablet_service.cc:1514] Processing DeleteTablet for tablet f53c958d83934f4dbbbdb8691f5b2969 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:12 UTC) from {username='oryx', principal='oryx/127.7.67.253@KRBTEST.COM'} at 127.0.0.1:37988
I20260806 18:30:12.252897 17611 tablet_service.cc:1514] Processing DeleteTablet for tablet f53c958d83934f4dbbbdb8691f5b2969 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:12 UTC) from {username='oryx', principal='oryx/127.7.67.253@KRBTEST.COM'} at 127.0.0.1:48624
I20260806 18:30:12.252950 18247 tablet_replica.cc:330] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536: stopping tablet replica
I20260806 18:30:12.253072 18247 raft_consensus.cc:2229] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:30:12.253203 18247 raft_consensus.cc:2258] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:12.253242 18249 tablet_replica.cc:330] T f53c958d83934f4dbbbdb8691f5b2969 P 04c0f654471b4bf19e477ac1f30b535b: stopping tablet replica
I20260806 18:30:12.253379 18249 raft_consensus.cc:2229] T f53c958d83934f4dbbbdb8691f5b2969 P 04c0f654471b4bf19e477ac1f30b535b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:30:12.253628 18249 raft_consensus.cc:2258] T f53c958d83934f4dbbbdb8691f5b2969 P 04c0f654471b4bf19e477ac1f30b535b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:12.254006 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 17398
I20260806 18:30:12.263345 18248 tablet_replica.cc:330] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44: stopping tablet replica
I20260806 18:30:12.263593 18248 raft_consensus.cc:2229] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:30:12.263746 18247 ts_tablet_manager.cc:1905] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:12.264117 18248 raft_consensus.cc:2258] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:12.264932 18248 ts_tablet_manager.cc:1905] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:12.266670 18247 ts_tablet_manager.cc:1918] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:30:12.266839 18247 log.cc:1196] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/f53c958d83934f4dbbbdb8691f5b2969
I20260806 18:30:12.267263 18247 ts_tablet_manager.cc:1939] T f53c958d83934f4dbbbdb8691f5b2969 P 09fb11cf02c9434fb56d3d390a1a3536: Deleting consensus metadata
I20260806 18:30:12.267601 18248 ts_tablet_manager.cc:1918] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:30:12.267716 18248 log.cc:1196] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/f53c958d83934f4dbbbdb8691f5b2969
I20260806 18:30:12.268008 18248 ts_tablet_manager.cc:1939] T f53c958d83934f4dbbbdb8691f5b2969 P 9a703950a82f49cb8d102abf747e0f44: Deleting consensus metadata
I20260806 18:30:12.268357 17195 catalog_manager.cc:4841] TS 09fb11cf02c9434fb56d3d390a1a3536 (127.7.67.195:45549): tablet f53c958d83934f4dbbbdb8691f5b2969 (table test-table [id=b2ab67e55f524e0eb3aa0629457fd7a6]) successfully deleted
I20260806 18:30:12.268729 17192 catalog_manager.cc:4841] TS 9a703950a82f49cb8d102abf747e0f44 (127.7.67.194:32907): tablet f53c958d83934f4dbbbdb8691f5b2969 (table test-table [id=b2ab67e55f524e0eb3aa0629457fd7a6]) successfully deleted
I20260806 18:30:12.271696 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 17712
I20260806 18:30:12.271739 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:12.271423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:45973 (local address 127.0.0.1:48632)
0806 18:30:12.271589 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:12.271646 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:45973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260806 18:30:12.275174 17195 catalog_manager.cc:4584] TS 04c0f654471b4bf19e477ac1f30b535b (127.7.67.193:45973): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f53c958d83934f4dbbbdb8691f5b2969: Network error: Client connection negotiation failed: client connection to 127.7.67.193:45973: connect: Connection refused (error 111)
I20260806 18:30:12.283582 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 17896
I20260806 18:30:12.301694 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 17027
I20260806 18:30:12.313478 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 17177
2026-08-06T18:30:12Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7715 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:30:12 dist-test-slave-5bn2 krb5kdc[18256](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:30:12 dist-test-slave-5bn2 krb5kdc[18256](info): set up 2 sockets
Aug 06 18:30:12 dist-test-slave-5bn2 krb5kdc[18256](info): commencing operation
krb5kdc: starting...
W20260806 18:30:13.437935 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:30:13 dist-test-slave-5bn2 krb5kdc[18256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:30:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:30:13Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.254.keytab.
Entry for principal oryx/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.254.keytab.
I20260806 18:30:13.566498 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:33367
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.254.keytab
--principal=oryx/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:40227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:33367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:13.677846 18271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:13.678114 18271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:13.678153 18271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:13.682037 18271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:30:13.682090 18271 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 18:30:13.682106 18271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:13.682121 18271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:13.682142 18271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:30:13.682160 18271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:30:13.687347 18271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:40227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:33367
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.254.keytab
--principal=oryx/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:33367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:13.688362 18271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:13.689323 18271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:30:13.697865 18271 server_base.cc:1034] running on GCE node
W20260806 18:30:13.697827 18278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:13.697826 18279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:13.697834 18283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:13.698740 18271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:13.699878 18271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:13.701045 18271 hybrid_clock.cc:648] HybridClock initialized: now 1786041013701029 us; error 28 us; skew 500 ppm
Aug 06 18:30:13 dist-test-slave-5bn2 krb5kdc[18256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041013, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:13.703816 18271 init.cc:377] Logged in from keytab as oryx/127.7.67.254@KRBTEST.COM (short username oryx)
I20260806 18:30:13.704703 18271 webserver.cc:460] Webserver started at http://127.7.67.254:37731/ using document root <none> and password file <none>
I20260806 18:30:13.705256 18271 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:13.705322 18271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:13.705513 18271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:13.707685 18271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "456b0ab10b6d40bb97f0e42f1c3baa1a"
format_stamp: "Formatted at 2026-08-06 18:30:13 on dist-test-slave-5bn2"
server_key: "08f7c3a8f9a82aa5df2cf3f768314e0e"
server_key_iv: "bfdc21ed3594ae5dd93b560e989a82b1"
server_key_version: "encryptionkey@0"
I20260806 18:30:13.708246 18271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "456b0ab10b6d40bb97f0e42f1c3baa1a"
format_stamp: "Formatted at 2026-08-06 18:30:13 on dist-test-slave-5bn2"
server_key: "08f7c3a8f9a82aa5df2cf3f768314e0e"
server_key_iv: "bfdc21ed3594ae5dd93b560e989a82b1"
server_key_version: "encryptionkey@0"
I20260806 18:30:13.712121 18271 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 18:30:13.714566 18292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:13.715860 18271 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:30:13.716002 18271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "456b0ab10b6d40bb97f0e42f1c3baa1a"
format_stamp: "Formatted at 2026-08-06 18:30:13 on dist-test-slave-5bn2"
server_key: "08f7c3a8f9a82aa5df2cf3f768314e0e"
server_key_iv: "bfdc21ed3594ae5dd93b560e989a82b1"
server_key_version: "encryptionkey@0"
I20260806 18:30:13.716113 18271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:13.735208 18271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:13.738581 18271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:13.738781 18271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:13.746867 18271 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:33367
I20260806 18:30:13.746892 18376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:33367 every 8 connection(s)
I20260806 18:30:13.747896 18271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:30:13.750748 18377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:13.752676 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 18271
I20260806 18:30:13.752795 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:30:13.753100 7439 external_mini_cluster.cc:1428] Setting key 22dde982d382008ff506d9dd421b6424
I20260806 18:30:13.757375 18377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a: Bootstrap starting.
I20260806 18:30:13.760216 18377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:13.761144 18377 log.cc:824] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:30:13 dist-test-slave-5bn2 krb5kdc[18256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM
I20260806 18:30:13.763154 18377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a: No bootstrap required, opened a new log
I20260806 18:30:13.766618 18377 raft_consensus.cc:348] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456b0ab10b6d40bb97f0e42f1c3baa1a" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 33367 } }
I20260806 18:30:13.766916 18377 raft_consensus.cc:374] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:13.766996 18377 raft_consensus.cc:729] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456b0ab10b6d40bb97f0e42f1c3baa1a, State: Initialized, Role: FOLLOWER
I20260806 18:30:13.767454 18377 consensus_queue.cc:260] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456b0ab10b6d40bb97f0e42f1c3baa1a" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 33367 } }
I20260806 18:30:13.767582 18377 raft_consensus.cc:388] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:30:13.767643 18377 raft_consensus.cc:482] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:30:13.767740 18377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:13.767931 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:13.754625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34944 (local address 127.7.67.254:33367)
0806 18:30:13.755151 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:13.755177 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:13.755222 (+ 45us) server_negotiation.cc:407] Connection header received
0806 18:30:13.756170 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:13.756195 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:13.756638 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:13.757013 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:13.757706 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:13.759218 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:13.760068 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:13.760769 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:13.760944 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:13.763323 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:13.763329 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:13.763342 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:13.763371 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:13.764957 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:13.765375 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:13.765381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:13.765386 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:13.765505 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:13.765770 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:13.765772 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:13.765774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:13.766195 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:13.766390 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:13.766668 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 18:30:13.767015 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":170,"threads_started":1}
I20260806 18:30:13.768698 18377 raft_consensus.cc:504] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456b0ab10b6d40bb97f0e42f1c3baa1a" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 33367 } }
I20260806 18:30:13.769028 18377 leader_election.cc:302] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 456b0ab10b6d40bb97f0e42f1c3baa1a; no voters:
I20260806 18:30:13.769323 18377 leader_election.cc:288] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:30:13.769425 18385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:13.769614 18385 raft_consensus.cc:686] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a [term 1 LEADER]: Becoming Leader. State: Replica: 456b0ab10b6d40bb97f0e42f1c3baa1a, State: Running, Role: LEADER
I20260806 18:30:13.769974 18385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456b0ab10b6d40bb97f0e42f1c3baa1a" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 33367 } }
I20260806 18:30:13.770691 18377 sys_catalog.cc:564] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:30:13.771657 18386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "456b0ab10b6d40bb97f0e42f1c3baa1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456b0ab10b6d40bb97f0e42f1c3baa1a" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 33367 } } }
I20260806 18:30:13.772022 18386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:13.771920 18387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 456b0ab10b6d40bb97f0e42f1c3baa1a. Latest consensus state: current_term: 1 leader_uuid: "456b0ab10b6d40bb97f0e42f1c3baa1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456b0ab10b6d40bb97f0e42f1c3baa1a" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 33367 } } }
I20260806 18:30:13.772339 18387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:13.773037 18395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:30:13.776082 18395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:30:13.781960 18395 catalog_manager.cc:1347] Generated new cluster ID: d8f4e8e7168d438aa5c31053d856dcf7
I20260806 18:30:13.782039 18395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:30:13.796572 18395 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:30:13.797580 18395 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:30:13.807883 18395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a: Generated new TSK 0
I20260806 18:30:13.808604 18395 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.193.keytab.
Entry for principal oryx/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.193.keytab.
I20260806 18:30:13.841320 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:33367
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.193.keytab
--principal=oryx/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:40227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:13.958299 18410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:13.958582 18410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:13.958636 18410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:13.962632 18410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:13.962699 18410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:13.962738 18410 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:30:13.962778 18410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:30:13.968098 18410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:40227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.193.keytab
--principal=oryx/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.67.254:33367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:13.969174 18410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:13.970223 18410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:13.979887 18415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:13.979885 18420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:13.979885 18416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:13.980230 18410 server_base.cc:1034] running on GCE node
I20260806 18:30:13.980690 18410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:13.981339 18410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:13.982498 18410 hybrid_clock.cc:648] HybridClock initialized: now 1786041013982481 us; error 30 us; skew 500 ppm
Aug 06 18:30:13 dist-test-slave-5bn2 krb5kdc[18256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041013, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:13.985317 18410 init.cc:377] Logged in from keytab as oryx/127.7.67.193@KRBTEST.COM (short username oryx)
I20260806 18:30:13.986217 18410 webserver.cc:460] Webserver started at http://127.7.67.193:44419/ using document root <none> and password file <none>
I20260806 18:30:13.986779 18410 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:13.986843 18410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:13.987085 18410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:13.989200 18410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "63537119fb62424fbe5fa9eddce11a4c"
format_stamp: "Formatted at 2026-08-06 18:30:13 on dist-test-slave-5bn2"
server_key: "7dc0efbf012a12175fee7c8727fa3820"
server_key_iv: "cfe23943ac4a25dcdcb3013a08caf986"
server_key_version: "encryptionkey@0"
I20260806 18:30:13.989759 18410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "63537119fb62424fbe5fa9eddce11a4c"
format_stamp: "Formatted at 2026-08-06 18:30:13 on dist-test-slave-5bn2"
server_key: "7dc0efbf012a12175fee7c8727fa3820"
server_key_iv: "cfe23943ac4a25dcdcb3013a08caf986"
server_key_version: "encryptionkey@0"
I20260806 18:30:13.993597 18410 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:30:13.996083 18429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:13.997304 18410 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:30:13.997407 18410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "63537119fb62424fbe5fa9eddce11a4c"
format_stamp: "Formatted at 2026-08-06 18:30:13 on dist-test-slave-5bn2"
server_key: "7dc0efbf012a12175fee7c8727fa3820"
server_key_iv: "cfe23943ac4a25dcdcb3013a08caf986"
server_key_version: "encryptionkey@0"
I20260806 18:30:13.997517 18410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:14.017457 18410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:14.020704 18410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:14.020869 18410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:14.022307 18410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:14.022363 18410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:14.022429 18410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:14.022460 18410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:14.031988 18410 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:33997
I20260806 18:30:14.032106 18591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:33997 every 8 connection(s)
Aug 06 18:30:14 dist-test-slave-5bn2 krb5kdc[18256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041013, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM
I20260806 18:30:14.032948 18410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:30:14.036247 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.023107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:41775 (local address 127.7.67.254:33367)
0806 18:30:14.023251 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.023256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.024177 (+ 921us) server_negotiation.cc:407] Connection header received
0806 18:30:14.025305 (+ 1128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.025309 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.025374 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.025482 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:14.027077 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.027909 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.029605 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.029898 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.030015 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.033003 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:14.033007 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:14.033010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:14.033041 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:14.034466 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:14.034951 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:14.034954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:14.034955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:14.035033 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:14.035310 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:14.035312 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:14.035313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:14.035504 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:14.035575 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:14.036001 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 18:30:14.036117 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:30:14.037118 18440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.023522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:33367 (local address 127.7.67.193:41775)
0806 18:30:14.023993 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:14.024056 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:14.025053 (+ 997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:14.025769 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:14.025777 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:14.026219 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:14.026923 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.026944 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.028051 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.028055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.029473 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.029485 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.030088 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.030094 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.030318 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.031044 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:14.031063 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:14.032822 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:14.034573 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:14.034578 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:14.034590 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:14.034839 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:14.035102 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:14.035104 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:14.035105 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:14.035237 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:14.035577 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:14.035581 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:14.035812 (+ 231us) client_negotiation.cc:769] Sending connection context
0806 18:30:14.035958 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 18:30:14.036223 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":114,"threads_started":1}
I20260806 18:30:14.038329 18593 heartbeater.cc:347] Connected to a master server at 127.7.67.254:33367
I20260806 18:30:14.038574 18593 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:14.038614 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 18410
I20260806 18:30:14.038744 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:30:14.039143 7439 external_mini_cluster.cc:1428] Setting key 57eac5952b00383d75c456ad0dd0120a
I20260806 18:30:14.039211 18593 heartbeater.cc:511] Master 127.7.67.254:33367 requested a full tablet report, sending...
I20260806 18:30:14.040802 18315 ts_manager.cc:194] Registered new tserver with Master: 63537119fb62424fbe5fa9eddce11a4c (127.7.67.193:33997)
I20260806 18:30:14.042023 18315 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:41775
I20260806 18:30:14.049566 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.042706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:41713 (local address 127.7.67.254:33367)
0806 18:30:14.042817 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.042821 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.042834 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:30:14.042888 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.042891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.042933 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.043009 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:14.043547 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.044236 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.045122 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.045455 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.045562 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.046590 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:14.046593 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:14.046595 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:14.046616 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:14.048033 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:14.048499 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:14.048502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:14.048503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:14.048577 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:14.048843 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:14.048845 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:14.048846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:14.049003 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:14.049071 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:14.049347 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 18:30:14.049441 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:30:14.049762 18600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.040980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:33367 (local address 127.7.67.193:41713)
0806 18:30:14.041285 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:14.041300 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:14.041383 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:14.043120 (+ 1737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:14.043123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:14.043311 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:14.043436 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.043442 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.044368 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.044372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.045000 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.045010 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.045568 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.045570 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.045636 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.046143 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:14.046155 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:14.046468 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:14.048150 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:14.048154 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:14.048156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:14.048409 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:14.048659 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:14.048662 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:14.048664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:14.048760 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:14.049091 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:14.049094 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:14.049201 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 18:30:14.049315 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 18:30:14.049468 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for oryx/127.7.67.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.194.keytab.
Entry for principal oryx/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.194.keytab.
I20260806 18:30:14.068075 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:33367
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.194.keytab
--principal=oryx/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:40227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:14.181385 18603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:14.181659 18603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:14.181722 18603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:14.185658 18603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:14.185720 18603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:14.185774 18603 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:30:14.185827 18603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:30:14.191114 18603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:40227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.194.keytab
--principal=oryx/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.67.254:33367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:14.192185 18603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:14.193225 18603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:14.202683 18613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:14.202718 18609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:14.202843 18603 server_base.cc:1034] running on GCE node
W20260806 18:30:14.203035 18610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:14.203452 18603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:14.204094 18603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:14.205269 18603 hybrid_clock.cc:648] HybridClock initialized: now 1786041014205252 us; error 26 us; skew 500 ppm
Aug 06 18:30:14 dist-test-slave-5bn2 krb5kdc[18256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041014, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:14.207965 18603 init.cc:377] Logged in from keytab as oryx/127.7.67.194@KRBTEST.COM (short username oryx)
I20260806 18:30:14.208876 18603 webserver.cc:460] Webserver started at http://127.7.67.194:44555/ using document root <none> and password file <none>
I20260806 18:30:14.209434 18603 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:14.209503 18603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:14.209712 18603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:14.211831 18603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "05c9707ab46349aebee2e0952c2574d5"
format_stamp: "Formatted at 2026-08-06 18:30:14 on dist-test-slave-5bn2"
server_key: "68a3dfa0ff0996fd23a206c326459824"
server_key_iv: "6e70be513ff168fb7142b223f28cdd66"
server_key_version: "encryptionkey@0"
I20260806 18:30:14.212383 18603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "05c9707ab46349aebee2e0952c2574d5"
format_stamp: "Formatted at 2026-08-06 18:30:14 on dist-test-slave-5bn2"
server_key: "68a3dfa0ff0996fd23a206c326459824"
server_key_iv: "6e70be513ff168fb7142b223f28cdd66"
server_key_version: "encryptionkey@0"
I20260806 18:30:14.216101 18603 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:30:14.218498 18622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:14.219817 18603 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:30:14.219941 18603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "05c9707ab46349aebee2e0952c2574d5"
format_stamp: "Formatted at 2026-08-06 18:30:14 on dist-test-slave-5bn2"
server_key: "68a3dfa0ff0996fd23a206c326459824"
server_key_iv: "6e70be513ff168fb7142b223f28cdd66"
server_key_version: "encryptionkey@0"
I20260806 18:30:14.220064 18603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:14.237458 18603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:14.240336 18603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:14.240522 18603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:14.242005 18603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:14.242056 18603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:14.242112 18603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:14.242165 18603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:30:14 dist-test-slave-5bn2 krb5kdc[18256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041014, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM
I20260806 18:30:14.252457 18603 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:43037
I20260806 18:30:14.252651 18771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:43037 every 8 connection(s)
I20260806 18:30:14.253578 18603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:30:14.254590 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 18603
I20260806 18:30:14.254709 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:30:14.255033 7439 external_mini_cluster.cc:1428] Setting key 4289f58ad523bcd709882ce90c6fb20e
I20260806 18:30:14.256265 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.242672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:39653 (local address 127.7.67.254:33367)
0806 18:30:14.242791 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.242794 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.243875 (+ 1081us) server_negotiation.cc:407] Connection header received
0806 18:30:14.244900 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.244904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.244974 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.245070 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:14.246457 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.247055 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.248770 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.249095 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.249182 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.252567 (+ 3385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:14.252571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:14.252574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:14.252602 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:14.254012 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:14.254627 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:14.254630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:14.254632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:14.254791 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:14.255146 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:14.255149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:14.255151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:14.255346 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:14.255439 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:14.255996 (+ 557us) server_negotiation.cc:299] Negotiation successful
0806 18:30:14.256116 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:30:14.256996 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.243229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:33367 (local address 127.7.67.194:39653)
0806 18:30:14.243715 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:14.243757 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:14.244671 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:14.245254 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:14.245263 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:14.245689 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:14.246316 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.246332 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.247178 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.247182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.248621 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.248635 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.249234 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.249239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.249442 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.250669 (+ 1227us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:14.250691 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:14.252398 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:14.254138 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:14.254144 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:14.254158 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:14.254517 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:14.254907 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:14.254910 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:14.254913 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:14.255053 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:14.255441 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:14.255446 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:14.255725 (+ 279us) client_negotiation.cc:769] Sending connection context
0806 18:30:14.255907 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 18:30:14.256166 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":137,"threads_started":1}
I20260806 18:30:14.258252 18773 heartbeater.cc:347] Connected to a master server at 127.7.67.254:33367
I20260806 18:30:14.258410 18773 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:14.259038 18773 heartbeater.cc:511] Master 127.7.67.254:33367 requested a full tablet report, sending...
I20260806 18:30:14.260039 18312 ts_manager.cc:194] Registered new tserver with Master: 05c9707ab46349aebee2e0952c2574d5 (127.7.67.194:43037)
I20260806 18:30:14.260705 18312 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:39653
WARNING: no policy specified for oryx/127.7.67.195@KRBTEST.COM; defaulting to no policy
I20260806 18:30:14.267379 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.260787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:47967 (local address 127.7.67.254:33367)
0806 18:30:14.260898 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.260902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.261117 (+ 215us) server_negotiation.cc:407] Connection header received
0806 18:30:14.261219 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.261222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.261264 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.261326 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:14.261758 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.262322 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.263156 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.263369 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.263453 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.264512 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:14.264516 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:14.264518 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:14.264543 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:14.265804 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:14.266187 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:14.266190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:14.266192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:14.266270 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:14.266513 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:14.266516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:14.266518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:14.266732 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:14.266819 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:14.267130 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 18:30:14.267232 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:30:14.267390 18782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.260713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:33367 (local address 127.7.67.194:47967)
0806 18:30:14.261030 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:14.261042 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:14.261138 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:14.261354 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:14.261357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:14.261556 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:14.261651 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.261657 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.262443 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.262446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.263044 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.263054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.263467 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.263471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.263570 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.264073 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:14.264083 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:14.264385 (+ 302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:14.265904 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:14.265907 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:14.265909 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:14.266100 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:14.266349 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:14.266352 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:14.266354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:14.266433 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:14.266827 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:14.266830 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:14.266959 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 18:30:14.267055 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 18:30:14.267190 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":98,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.195.keytab.
Entry for principal oryx/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.195.keytab.
I20260806 18:30:14.283378 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:33367
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.195.keytab
--principal=oryx/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:40227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:14.397207 18784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:14.397459 18784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:14.397502 18784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:14.401476 18784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:14.401535 18784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:14.401578 18784 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:30:14.401630 18784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:30:14.406831 18784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:40227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.195.keytab
--principal=oryx/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.67.254:33367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:14.407904 18784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:14.408881 18784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:14.418223 18794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:14.418324 18792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:14.418226 18791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:14.418460 18784 server_base.cc:1034] running on GCE node
I20260806 18:30:14.418937 18784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:14.419638 18784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:14.420801 18784 hybrid_clock.cc:648] HybridClock initialized: now 1786041014420783 us; error 35 us; skew 500 ppm
Aug 06 18:30:14 dist-test-slave-5bn2 krb5kdc[18256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041014, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:14.423537 18784 init.cc:377] Logged in from keytab as oryx/127.7.67.195@KRBTEST.COM (short username oryx)
I20260806 18:30:14.424464 18784 webserver.cc:460] Webserver started at http://127.7.67.195:32859/ using document root <none> and password file <none>
I20260806 18:30:14.425019 18784 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:14.425096 18784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:14.425304 18784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:14.427408 18784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "ef0e172af3fb4dc3a3befe1f5068af94"
format_stamp: "Formatted at 2026-08-06 18:30:14 on dist-test-slave-5bn2"
server_key: "31e2867384afe4a4d35dcf4797c6a74b"
server_key_iv: "c83821a239ee80b06115766c594934f9"
server_key_version: "encryptionkey@0"
I20260806 18:30:14.427949 18784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "ef0e172af3fb4dc3a3befe1f5068af94"
format_stamp: "Formatted at 2026-08-06 18:30:14 on dist-test-slave-5bn2"
server_key: "31e2867384afe4a4d35dcf4797c6a74b"
server_key_iv: "c83821a239ee80b06115766c594934f9"
server_key_version: "encryptionkey@0"
I20260806 18:30:14.431669 18784 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:30:14.433965 18802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:14.435173 18784 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:30:14.435294 18784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "ef0e172af3fb4dc3a3befe1f5068af94"
format_stamp: "Formatted at 2026-08-06 18:30:14 on dist-test-slave-5bn2"
server_key: "31e2867384afe4a4d35dcf4797c6a74b"
server_key_iv: "c83821a239ee80b06115766c594934f9"
server_key_version: "encryptionkey@0"
I20260806 18:30:14.435390 18784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:14.455951 18784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:14.459220 18784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:14.459398 18784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:14.460945 18784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:14.460996 18784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:14.461061 18784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:14.461089 18784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:30:14 dist-test-slave-5bn2 krb5kdc[18256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041014, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM
I20260806 18:30:14.471155 18784 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:37097
I20260806 18:30:14.471230 18967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:37097 every 8 connection(s)
I20260806 18:30:14.472108 18784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:30:14.474834 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.461637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:43543 (local address 127.7.67.254:33367)
0806 18:30:14.461794 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.461800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.462619 (+ 819us) server_negotiation.cc:407] Connection header received
0806 18:30:14.463595 (+ 976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.463599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.463656 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.463709 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:14.465344 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.466146 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.467923 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.468237 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.468321 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.471243 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:14.471247 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:14.471250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:14.471279 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:14.472845 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:14.473385 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:14.473389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:14.473391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:14.473473 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:14.473786 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:14.473790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:14.473792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:14.473987 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:14.474071 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:14.474553 (+ 482us) server_negotiation.cc:299] Negotiation successful
0806 18:30:14.474673 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:30:14.475615 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.462003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:33367 (local address 127.7.67.195:43543)
0806 18:30:14.462447 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:14.462508 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:14.463363 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:14.464122 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:14.464131 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:14.464595 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:14.465200 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.465221 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.466296 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.466301 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.467787 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.467800 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.468397 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.468402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.468626 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.469268 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:14.469289 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:14.471082 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:14.472947 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:14.472953 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:14.472967 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:14.473285 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:14.473552 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:14.473555 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:14.473558 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:14.473692 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:14.474085 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:14.474092 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:14.474320 (+ 228us) client_negotiation.cc:769] Sending connection context
0806 18:30:14.474483 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 18:30:14.474786 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":112,"threads_started":1}
I20260806 18:30:14.476816 18970 heartbeater.cc:347] Connected to a master server at 127.7.67.254:33367
I20260806 18:30:14.476959 18970 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:14.477425 18970 heartbeater.cc:511] Master 127.7.67.254:33367 requested a full tablet report, sending...
I20260806 18:30:14.478626 18315 ts_manager.cc:194] Registered new tserver with Master: ef0e172af3fb4dc3a3befe1f5068af94 (127.7.67.195:37097)
I20260806 18:30:14.479334 18315 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:43543
I20260806 18:30:14.480629 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 18784
I20260806 18:30:14.480770 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:30:14.481120 7439 external_mini_cluster.cc:1428] Setting key 1bc8ac59ae85ce8ef977e56dbdec8d61
I20260806 18:30:14.483033 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:30:14.485723 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.478979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:33913 (local address 127.7.67.254:33367)
0806 18:30:14.479083 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.479087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.479318 (+ 231us) server_negotiation.cc:407] Connection header received
0806 18:30:14.479423 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.479426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.479472 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.479536 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:14.479997 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.480542 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.481592 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.481812 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.481897 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.482929 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:14.482933 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:14.482935 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:14.482959 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:14.484255 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:14.484645 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:14.484649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:14.484650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:14.484727 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:14.484970 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:14.484974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:14.484976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:14.485145 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:14.485228 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:14.485485 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 18:30:14.485586 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 18:30:14.485735 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.478950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:33367 (local address 127.7.67.195:33913)
0806 18:30:14.479233 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:14.479244 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:14.479340 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:14.479554 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:14.479556 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:14.479768 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:14.479886 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.479894 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.480693 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.480696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.481480 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.481490 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.481908 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.481912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.482009 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.482456 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:14.482470 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:14.482774 (+ 304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:14.484356 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:14.484360 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:14.484362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:14.484557 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:14.484807 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:14.484810 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:14.484812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:14.484892 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:14.485234 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:14.485237 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:14.485326 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 18:30:14.485413 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 18:30:14.485554 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":126,"threads_started":1}
Aug 06 18:30:14 dist-test-slave-5bn2 krb5kdc[18256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041014, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:30:14 dist-test-slave-5bn2 krb5kdc[18256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:14 dist-test-slave-5bn2 krb5kdc[18256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:30:14.508056 18381 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:14.502088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34950 (local address 127.7.67.254:33367)
0806 18:30:14.502213 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.502217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.502230 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:30:14.502273 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.502276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.502318 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.502385 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:14.503112 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.503638 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.504479 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.504710 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.504788 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.507915 (+ 3127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34950: BlockingRecv error: recv got EOF from 127.0.0.1:34950 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 18:30:14 dist-test-slave-5bn2 krb5kdc[18256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041014, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM
I20260806 18:30:14.517122 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.509422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34964 (local address 127.7.67.254:33367)
0806 18:30:14.509566 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.509570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.509657 (+ 87us) server_negotiation.cc:407] Connection header received
0806 18:30:14.509793 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.509796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.509836 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.509897 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:14.510515 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.511091 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.511848 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.512052 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.512140 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.514342 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:14.514345 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:14.514347 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:14.514380 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:14.515571 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:14.515967 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:14.515970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:14.515971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:14.516049 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:14.516297 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:14.516301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:14.516303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:14.516495 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:14.516620 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:14.516849 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 18:30:14.516963 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:30:14.519966 18315 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:30:14.523047 18315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:30:14.537935 19004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.532631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:33997 (local address 127.0.0.1:49932)
0806 18:30:14.532906 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:14.532918 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:14.533002 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:14.533835 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:14.533838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:14.533852 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:14.533986 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.533993 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.535123 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.535126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.536524 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.536535 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.537380 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.537384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.537489 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.537530 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 18:30:14.537642 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 18:30:14.537773 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":40,"thread_start_us":80,"threads_started":1}
I20260806 18:30:14.537925 19007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.532805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49932 (local address 127.7.67.193:33997)
0806 18:30:14.533216 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.533231 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.533254 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:30:14.533338 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.533343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.533537 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.533692 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:14.534118 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.535017 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.536652 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.537264 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.537362 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.537554 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:14.537690 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 18:30:14.537781 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":92,"threads_started":1}
I20260806 18:30:14.538221 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.531928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:43037 (local address 127.0.0.1:45106)
0806 18:30:14.532267 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:14.532314 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:14.532441 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:14.533709 (+ 1268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:14.533716 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:14.533754 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:14.534044 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.534053 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.535549 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.535552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.536716 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.536727 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.537798 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.537802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.537887 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.537959 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 18:30:14.538007 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 18:30:14.538078 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":174,"spinlock_wait_cycles":3968,"thread_start_us":80,"threads_started":1}
I20260806 18:30:14.538453 19001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.532149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:37097 (local address 127.0.0.1:47776)
0806 18:30:14.532690 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:14.532704 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:14.532798 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:14.533710 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:14.533716 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:14.533751 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:14.534012 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.534022 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.535525 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.535529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.536774 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.536783 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.538039 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.538043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.538203 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.538225 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:30:14.538275 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 18:30:14.538336 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":210,"spinlock_wait_cycles":8576,"thread_start_us":98,"threads_started":1}
I20260806 18:30:14.538872 19000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.532115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45106 (local address 127.7.67.194:43037)
0806 18:30:14.533180 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.533196 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.533223 (+ 27us) server_negotiation.cc:407] Connection header received
0806 18:30:14.533308 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.533314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.533534 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.534055 (+ 521us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:14.534201 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.535399 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.536839 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.537659 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.538324 (+ 665us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.538554 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:14.538610 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 18:30:14.538707 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":835,"thread_start_us":117,"threads_started":1}
I20260806 18:30:14.539270 19005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.532649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47776 (local address 127.7.67.195:37097)
0806 18:30:14.533235 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.533246 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.533264 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:30:14.533338 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.533343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.533543 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.533823 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:14.534149 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.535398 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.536901 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.537899 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.538793 (+ 894us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.538989 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:14.539042 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 18:30:14.539134 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":104,"threads_started":1}
I20260806 18:30:14.541538 18879 tablet_service.cc:1467] Processing CreateTablet for tablet 50f8d524aa6240e0a49db54865b71e30 (DEFAULT_TABLE table=test-table [id=e73884b3931243f29d59ff91e6626268]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:14.541553 18690 tablet_service.cc:1467] Processing CreateTablet for tablet 50f8d524aa6240e0a49db54865b71e30 (DEFAULT_TABLE table=test-table [id=e73884b3931243f29d59ff91e6626268]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:14.541550 18497 tablet_service.cc:1467] Processing CreateTablet for tablet 50f8d524aa6240e0a49db54865b71e30 (DEFAULT_TABLE table=test-table [id=e73884b3931243f29d59ff91e6626268]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:14.542596 18690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50f8d524aa6240e0a49db54865b71e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:14.542596 18497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50f8d524aa6240e0a49db54865b71e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:14.542614 18879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50f8d524aa6240e0a49db54865b71e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:14.549259 19008 tablet_bootstrap.cc:492] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94: Bootstrap starting.
I20260806 18:30:14.550536 19009 tablet_bootstrap.cc:492] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c: Bootstrap starting.
I20260806 18:30:14.551175 19010 tablet_bootstrap.cc:492] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5: Bootstrap starting.
I20260806 18:30:14.551846 19008 tablet_bootstrap.cc:654] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:14.552570 19009 tablet_bootstrap.cc:654] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:14.552561 19008 log.cc:824] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:14.553567 19010 tablet_bootstrap.cc:654] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:14.553754 19009 log.cc:824] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:14.554318 19010 log.cc:824] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:14.554457 19008 tablet_bootstrap.cc:492] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94: No bootstrap required, opened a new log
I20260806 18:30:14.554699 19008 ts_tablet_manager.cc:1397] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:30:14.555734 19009 tablet_bootstrap.cc:492] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c: No bootstrap required, opened a new log
I20260806 18:30:14.555943 19009 ts_tablet_manager.cc:1397] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:30:14.556351 19010 tablet_bootstrap.cc:492] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5: No bootstrap required, opened a new log
I20260806 18:30:14.556541 19010 ts_tablet_manager.cc:1397] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:30:14.558281 19008 raft_consensus.cc:348] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.558524 19008 raft_consensus.cc:374] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:14.558598 19008 raft_consensus.cc:729] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef0e172af3fb4dc3a3befe1f5068af94, State: Initialized, Role: FOLLOWER
I20260806 18:30:14.559075 19008 consensus_queue.cc:260] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.559746 18970 heartbeater.cc:503] Master 127.7.67.254:33367 was elected leader, sending a full tablet report...
I20260806 18:30:14.560133 19008 ts_tablet_manager.cc:1428] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:30:14.559950 19009 raft_consensus.cc:348] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.560205 19009 raft_consensus.cc:374] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:14.560253 19009 raft_consensus.cc:729] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63537119fb62424fbe5fa9eddce11a4c, State: Initialized, Role: FOLLOWER
I20260806 18:30:14.560391 19010 raft_consensus.cc:348] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.560636 19010 raft_consensus.cc:374] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:14.560724 19010 raft_consensus.cc:729] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05c9707ab46349aebee2e0952c2574d5, State: Initialized, Role: FOLLOWER
I20260806 18:30:14.560758 19009 consensus_queue.cc:260] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.561165 19010 consensus_queue.cc:260] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.561417 18593 heartbeater.cc:503] Master 127.7.67.254:33367 was elected leader, sending a full tablet report...
I20260806 18:30:14.561784 19009 ts_tablet_manager.cc:1428] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:30:14.562090 19010 ts_tablet_manager.cc:1428] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:30:14.562383 18773 heartbeater.cc:503] Master 127.7.67.254:33367 was elected leader, sending a full tablet report...
I20260806 18:30:14.576362 19018 raft_consensus.cc:482] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:30:14.576560 19018 raft_consensus.cc:504] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.577816 19018 leader_election.cc:288] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63537119fb62424fbe5fa9eddce11a4c (127.7.67.193:33997), ef0e172af3fb4dc3a3befe1f5068af94 (127.7.67.195:37097)
I20260806 18:30:14.581686 19007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.578323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:38419 (local address 127.7.67.193:33997)
0806 18:30:14.578464 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.578469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.578483 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:30:14.578537 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.578540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.578590 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.578659 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:14.578954 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.579538 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.580591 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.581139 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.581227 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.581269 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:14.581481 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 18:30:14.581545 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 18:30:14.581697 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.578035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:33997 (local address 127.7.67.194:38419)
0806 18:30:14.578226 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:14.578240 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:14.578330 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:14.578690 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:14.578693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:14.578712 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:14.578838 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.578866 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.579639 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.579643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.580444 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.580453 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.581241 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.581244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.581350 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.581376 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:30:14.581451 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 18:30:14.581544 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":3}
I20260806 18:30:14.582085 19005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.578180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:35635 (local address 127.7.67.195:37097)
0806 18:30:14.578315 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.578320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.578584 (+ 264us) server_negotiation.cc:407] Connection header received
0806 18:30:14.578693 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.578695 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.578742 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.578817 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:14.579110 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.579596 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.581015 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.581507 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.581615 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.581659 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:14.581881 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 18:30:14.581971 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:30:14.582432 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.578087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:37097 (local address 127.7.67.194:35635)
0806 18:30:14.578490 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:14.578503 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:14.578602 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:14.578834 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:14.578835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:14.578873 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:14.579012 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.579018 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.579705 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.579708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.580879 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.580900 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.581632 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.581636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.581733 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.581757 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:30:14.582230 (+ 473us) client_negotiation.cc:240] Negotiation successful
0806 18:30:14.582302 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":129,"threads_started":1}
I20260806 18:30:14.582376 18532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50f8d524aa6240e0a49db54865b71e30" candidate_uuid: "05c9707ab46349aebee2e0952c2574d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63537119fb62424fbe5fa9eddce11a4c" is_pre_election: true
I20260806 18:30:14.582712 18532 raft_consensus.cc:2454] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05c9707ab46349aebee2e0952c2574d5 in term 0.
I20260806 18:30:14.583091 18909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50f8d524aa6240e0a49db54865b71e30" candidate_uuid: "05c9707ab46349aebee2e0952c2574d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" is_pre_election: true
I20260806 18:30:14.583263 18625 leader_election.cc:302] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05c9707ab46349aebee2e0952c2574d5, 63537119fb62424fbe5fa9eddce11a4c; no voters:
I20260806 18:30:14.583393 18909 raft_consensus.cc:2454] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05c9707ab46349aebee2e0952c2574d5 in term 0.
I20260806 18:30:14.583557 19018 raft_consensus.cc:2781] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:30:14.583647 19018 raft_consensus.cc:482] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:30:14.583704 19018 raft_consensus.cc:3037] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:14.584755 19018 raft_consensus.cc:504] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.585080 19018 leader_election.cc:288] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [CANDIDATE]: Term 1 election: Requested vote from peers 63537119fb62424fbe5fa9eddce11a4c (127.7.67.193:33997), ef0e172af3fb4dc3a3befe1f5068af94 (127.7.67.195:37097)
I20260806 18:30:14.585415 18909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50f8d524aa6240e0a49db54865b71e30" candidate_uuid: "05c9707ab46349aebee2e0952c2574d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef0e172af3fb4dc3a3befe1f5068af94"
I20260806 18:30:14.585426 18532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50f8d524aa6240e0a49db54865b71e30" candidate_uuid: "05c9707ab46349aebee2e0952c2574d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63537119fb62424fbe5fa9eddce11a4c"
I20260806 18:30:14.585546 18909 raft_consensus.cc:3037] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:14.585546 18532 raft_consensus.cc:3037] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:14.586835 18532 raft_consensus.cc:2454] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05c9707ab46349aebee2e0952c2574d5 in term 1.
I20260806 18:30:14.586835 18909 raft_consensus.cc:2454] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05c9707ab46349aebee2e0952c2574d5 in term 1.
I20260806 18:30:14.587164 18629 leader_election.cc:302] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05c9707ab46349aebee2e0952c2574d5, ef0e172af3fb4dc3a3befe1f5068af94; no voters:
I20260806 18:30:14.587327 19018 raft_consensus.cc:2781] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:14.587548 19018 raft_consensus.cc:686] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [term 1 LEADER]: Becoming Leader. State: Replica: 05c9707ab46349aebee2e0952c2574d5, State: Running, Role: LEADER
I20260806 18:30:14.587857 19018 consensus_queue.cc:237] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.591157 18312 catalog_manager.cc:5495] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05c9707ab46349aebee2e0952c2574d5 (127.7.67.194). New cstate: current_term: 1 leader_uuid: "05c9707ab46349aebee2e0952c2574d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } health_report { overall_health: UNKNOWN } } }
I20260806 18:30:14.603307 19000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.600002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45120 (local address 127.7.67.194:43037)
0806 18:30:14.600115 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.600119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.600132 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:30:14.600255 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.600257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.600310 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.600394 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:30:14.600577 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.601090 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.601961 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.602176 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.602250 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.602520 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:30:14.603004 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:30:14.603085 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:14.603138 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 18:30:14.603193 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:30:14.609349 18532 raft_consensus.cc:1261] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c [term 1 FOLLOWER]: Refusing update from remote peer 05c9707ab46349aebee2e0952c2574d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:30:14.609740 18909 raft_consensus.cc:1261] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94 [term 1 FOLLOWER]: Refusing update from remote peer 05c9707ab46349aebee2e0952c2574d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:30:14.610085 19018 consensus_queue.cc:1035] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:30:14.610368 19020 consensus_queue.cc:1035] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:30:14.617992 19023 mvcc.cc:205] Tried to move back new op lower bound from 7315623995832057856 to 7315623995754504192. Current Snapshot: MvccSnapshot[applied={T|T < 7315623995832057856}]
I20260806 18:30:14.618877 19026 mvcc.cc:205] Tried to move back new op lower bound from 7315623995832057856 to 7315623995754504192. Current Snapshot: MvccSnapshot[applied={T|T < 7315623995832057856}]
I20260806 18:30:14.627877 18311 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:30:14.628039 18311 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:30:14.630417 18311 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a: Sending DeleteTablet for 3 replicas of tablet 50f8d524aa6240e0a49db54865b71e30
I20260806 18:30:14.631227 18690 tablet_service.cc:1514] Processing DeleteTablet for tablet 50f8d524aa6240e0a49db54865b71e30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:14 UTC) from {username='oryx', principal='oryx/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:45106
I20260806 18:30:14.631251 18879 tablet_service.cc:1514] Processing DeleteTablet for tablet 50f8d524aa6240e0a49db54865b71e30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:14 UTC) from {username='oryx', principal='oryx/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:47776
I20260806 18:30:14.631520 18497 tablet_service.cc:1514] Processing DeleteTablet for tablet 50f8d524aa6240e0a49db54865b71e30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:14 UTC) from {username='oryx', principal='oryx/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:49932
I20260806 18:30:14.631768 19036 tablet_replica.cc:330] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94: stopping tablet replica
I20260806 18:30:14.632025 19035 tablet_replica.cc:330] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5: stopping tablet replica
I20260806 18:30:14.632143 19036 raft_consensus.cc:2229] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:30:14.632375 19035 raft_consensus.cc:2229] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:30:14.632572 19036 raft_consensus.cc:2258] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:14.632647 18311 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:30:14.633062 18311 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:30:14.633301 19038 tablet_replica.cc:330] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c: stopping tablet replica
I20260806 18:30:14.633574 19038 raft_consensus.cc:2229] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:30:14.633838 19038 raft_consensus.cc:2258] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:14.633963 19036 ts_tablet_manager.cc:1905] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:14.635238 19038 ts_tablet_manager.cc:1905] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:14.636739 19036 ts_tablet_manager.cc:1918] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:30:14.636922 19036 log.cc:1196] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/50f8d524aa6240e0a49db54865b71e30
I20260806 18:30:14.637300 19036 ts_tablet_manager.cc:1939] T 50f8d524aa6240e0a49db54865b71e30 P ef0e172af3fb4dc3a3befe1f5068af94: Deleting consensus metadata
I20260806 18:30:14.637887 19038 ts_tablet_manager.cc:1918] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:30:14.638067 19038 log.cc:1196] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/wal/wals/50f8d524aa6240e0a49db54865b71e30
I20260806 18:30:14.638491 19038 ts_tablet_manager.cc:1939] T 50f8d524aa6240e0a49db54865b71e30 P 63537119fb62424fbe5fa9eddce11a4c: Deleting consensus metadata
I20260806 18:30:14.638821 18296 catalog_manager.cc:4841] TS ef0e172af3fb4dc3a3befe1f5068af94 (127.7.67.195:37097): tablet 50f8d524aa6240e0a49db54865b71e30 (table test-table [id=e73884b3931243f29d59ff91e6626268]) successfully deleted
I20260806 18:30:14.639178 18879 tablet_service.cc:1467] Processing CreateTablet for tablet c35ca7f730aa42bfa0c82fec6c20fe67 (DEFAULT_TABLE table=test-table [id=cbaaf06bcc2e4adb8232376e8fbd9da1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:14.639464 18879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c35ca7f730aa42bfa0c82fec6c20fe67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:14.640157 18293 catalog_manager.cc:4841] TS 63537119fb62424fbe5fa9eddce11a4c (127.7.67.193:33997): tablet 50f8d524aa6240e0a49db54865b71e30 (table test-table [id=e73884b3931243f29d59ff91e6626268]) successfully deleted
I20260806 18:30:14.640224 18497 tablet_service.cc:1467] Processing CreateTablet for tablet c35ca7f730aa42bfa0c82fec6c20fe67 (DEFAULT_TABLE table=test-table [id=cbaaf06bcc2e4adb8232376e8fbd9da1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:14.640489 18497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c35ca7f730aa42bfa0c82fec6c20fe67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:14.641566 19035 raft_consensus.cc:2258] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:14.641758 19008 tablet_bootstrap.cc:492] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94: Bootstrap starting.
I20260806 18:30:14.642578 19009 tablet_bootstrap.cc:492] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c: Bootstrap starting.
I20260806 18:30:14.639905 18690 tablet_service.cc:1467] Processing CreateTablet for tablet c35ca7f730aa42bfa0c82fec6c20fe67 (DEFAULT_TABLE table=test-table [id=cbaaf06bcc2e4adb8232376e8fbd9da1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:14.642727 19008 tablet_bootstrap.cc:654] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:14.642876 18690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c35ca7f730aa42bfa0c82fec6c20fe67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:14.643499 19009 tablet_bootstrap.cc:654] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:14.643477 19035 ts_tablet_manager.cc:1905] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:14.643898 19008 tablet_bootstrap.cc:492] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94: No bootstrap required, opened a new log
I20260806 18:30:14.643985 19008 ts_tablet_manager.cc:1397] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:30:14.644398 19008 raft_consensus.cc:348] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.644516 19008 raft_consensus.cc:374] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:14.644548 19008 raft_consensus.cc:729] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef0e172af3fb4dc3a3befe1f5068af94, State: Initialized, Role: FOLLOWER
I20260806 18:30:14.644665 19008 consensus_queue.cc:260] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.644930 19008 ts_tablet_manager.cc:1428] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:30:14.645325 19010 tablet_bootstrap.cc:492] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5: Bootstrap starting.
I20260806 18:30:14.645926 19035 ts_tablet_manager.cc:1918] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:30:14.646050 19035 log.cc:1196] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/50f8d524aa6240e0a49db54865b71e30
I20260806 18:30:14.646257 19010 tablet_bootstrap.cc:654] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:14.646379 19035 ts_tablet_manager.cc:1939] T 50f8d524aa6240e0a49db54865b71e30 P 05c9707ab46349aebee2e0952c2574d5: Deleting consensus metadata
I20260806 18:30:14.647512 18293 catalog_manager.cc:4841] TS 05c9707ab46349aebee2e0952c2574d5 (127.7.67.194:43037): tablet 50f8d524aa6240e0a49db54865b71e30 (table test-table [id=e73884b3931243f29d59ff91e6626268]) successfully deleted
I20260806 18:30:14.648041 19010 tablet_bootstrap.cc:492] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5: No bootstrap required, opened a new log
I20260806 18:30:14.648068 19009 tablet_bootstrap.cc:492] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c: No bootstrap required, opened a new log
I20260806 18:30:14.648133 19010 ts_tablet_manager.cc:1397] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:30:14.648141 19009 ts_tablet_manager.cc:1397] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260806 18:30:14.648537 19009 raft_consensus.cc:348] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.648645 19009 raft_consensus.cc:374] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:14.648584 19010 raft_consensus.cc:348] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.648679 19009 raft_consensus.cc:729] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63537119fb62424fbe5fa9eddce11a4c, State: Initialized, Role: FOLLOWER
I20260806 18:30:14.648698 19010 raft_consensus.cc:374] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:14.648729 19010 raft_consensus.cc:729] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05c9707ab46349aebee2e0952c2574d5, State: Initialized, Role: FOLLOWER
I20260806 18:30:14.648790 19009 consensus_queue.cc:260] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.648837 19010 consensus_queue.cc:260] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.649072 19010 ts_tablet_manager.cc:1428] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:30:14.649081 19009 ts_tablet_manager.cc:1428] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260806 18:30:14.723176 18971 tablet.cc:2369] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:30:14.754776 18775 tablet.cc:2369] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:30:14.784699 18594 tablet.cc:2369] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:30:14.825822 19018 raft_consensus.cc:482] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:30:14.825980 19018 raft_consensus.cc:504] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.826447 19018 leader_election.cc:288] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63537119fb62424fbe5fa9eddce11a4c (127.7.67.193:33997), ef0e172af3fb4dc3a3befe1f5068af94 (127.7.67.195:37097)
I20260806 18:30:14.826990 18909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c35ca7f730aa42bfa0c82fec6c20fe67" candidate_uuid: "05c9707ab46349aebee2e0952c2574d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" is_pre_election: true
I20260806 18:30:14.826990 18532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c35ca7f730aa42bfa0c82fec6c20fe67" candidate_uuid: "05c9707ab46349aebee2e0952c2574d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63537119fb62424fbe5fa9eddce11a4c" is_pre_election: true
I20260806 18:30:14.827159 18909 raft_consensus.cc:2454] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05c9707ab46349aebee2e0952c2574d5 in term 0.
I20260806 18:30:14.827159 18532 raft_consensus.cc:2454] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05c9707ab46349aebee2e0952c2574d5 in term 0.
I20260806 18:30:14.827488 18625 leader_election.cc:302] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 05c9707ab46349aebee2e0952c2574d5, 63537119fb62424fbe5fa9eddce11a4c, ef0e172af3fb4dc3a3befe1f5068af94; no voters:
I20260806 18:30:14.827690 19018 raft_consensus.cc:2781] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:30:14.827754 19018 raft_consensus.cc:482] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:30:14.827780 19018 raft_consensus.cc:3037] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:14.828804 19018 raft_consensus.cc:504] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.829145 19018 leader_election.cc:288] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [CANDIDATE]: Term 1 election: Requested vote from peers 63537119fb62424fbe5fa9eddce11a4c (127.7.67.193:33997), ef0e172af3fb4dc3a3befe1f5068af94 (127.7.67.195:37097)
I20260806 18:30:14.829653 18532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c35ca7f730aa42bfa0c82fec6c20fe67" candidate_uuid: "05c9707ab46349aebee2e0952c2574d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63537119fb62424fbe5fa9eddce11a4c"
I20260806 18:30:14.829653 18909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c35ca7f730aa42bfa0c82fec6c20fe67" candidate_uuid: "05c9707ab46349aebee2e0952c2574d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef0e172af3fb4dc3a3befe1f5068af94"
I20260806 18:30:14.829789 18909 raft_consensus.cc:3037] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:14.829789 18532 raft_consensus.cc:3037] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:14.830991 18909 raft_consensus.cc:2454] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05c9707ab46349aebee2e0952c2574d5 in term 1.
I20260806 18:30:14.830991 18532 raft_consensus.cc:2454] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05c9707ab46349aebee2e0952c2574d5 in term 1.
I20260806 18:30:14.831319 18629 leader_election.cc:302] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05c9707ab46349aebee2e0952c2574d5, ef0e172af3fb4dc3a3befe1f5068af94; no voters:
I20260806 18:30:14.831525 19018 raft_consensus.cc:2781] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:14.831622 19018 raft_consensus.cc:686] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [term 1 LEADER]: Becoming Leader. State: Replica: 05c9707ab46349aebee2e0952c2574d5, State: Running, Role: LEADER
I20260806 18:30:14.831761 19018 consensus_queue.cc:237] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.833425 18312 catalog_manager.cc:5495] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05c9707ab46349aebee2e0952c2574d5 (127.7.67.194). New cstate: current_term: 1 leader_uuid: "05c9707ab46349aebee2e0952c2574d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } health_report { overall_health: UNKNOWN } } }
Aug 06 18:30:14 dist-test-slave-5bn2 krb5kdc[18256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041013, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.254@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM
I20260806 18:30:14.864472 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.856707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34976 (local address 127.7.67.254:33367)
0806 18:30:14.856841 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.856846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.857037 (+ 191us) server_negotiation.cc:407] Connection header received
0806 18:30:14.857161 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.857163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.857208 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.857267 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:14.857954 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.858562 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.859373 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.859594 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.859675 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.861912 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:14.861915 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:14.861917 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:14.861939 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:14.863078 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:14.863472 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:14.863474 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:14.863475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:14.863540 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:14.863727 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:14.863728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:14.863730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:14.863897 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:14.863965 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:14.864208 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 18:30:14.864336 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":46}
I20260806 18:30:14.864475 19055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.856648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:33367 (local address 127.0.0.1:34976)
0806 18:30:14.856945 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:14.856958 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:14.857074 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:14.857293 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:14.857295 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:14.857698 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:14.857812 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.857819 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.858662 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.858665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.859268 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.859277 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.859686 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.859689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.859784 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.860348 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:14.860360 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:14.861769 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:14.863161 (+ 1392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:14.863170 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:14.863176 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:14.863397 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:14.863597 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:14.863598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:14.863600 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:14.863670 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:14.863974 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:14.863978 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:14.864082 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 18:30:14.864167 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 18:30:14.864285 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":24,"thread_start_us":122,"threads_started":1}
I20260806 18:30:14.868383 18312 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:34976:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 18:30:14.869107 18312 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:30:14.873910 18690 tablet_service.cc:1467] Processing CreateTablet for tablet 578df3dca0b349d48291c750c77800b8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0c4f2474320c46a9a65fef035f689728]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:30:14.873910 18497 tablet_service.cc:1467] Processing CreateTablet for tablet 578df3dca0b349d48291c750c77800b8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0c4f2474320c46a9a65fef035f689728]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:30:14.873980 18879 tablet_service.cc:1467] Processing CreateTablet for tablet 578df3dca0b349d48291c750c77800b8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0c4f2474320c46a9a65fef035f689728]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:30:14.874334 18497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 578df3dca0b349d48291c750c77800b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:14.874336 18690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 578df3dca0b349d48291c750c77800b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:14.874414 18879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 578df3dca0b349d48291c750c77800b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:14.877167 19008 tablet_bootstrap.cc:492] T 578df3dca0b349d48291c750c77800b8 P ef0e172af3fb4dc3a3befe1f5068af94: Bootstrap starting.
I20260806 18:30:14.878101 19008 tablet_bootstrap.cc:654] T 578df3dca0b349d48291c750c77800b8 P ef0e172af3fb4dc3a3befe1f5068af94: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:14.878710 19009 tablet_bootstrap.cc:492] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c: Bootstrap starting.
I20260806 18:30:14.879282 19008 tablet_bootstrap.cc:492] T 578df3dca0b349d48291c750c77800b8 P ef0e172af3fb4dc3a3befe1f5068af94: No bootstrap required, opened a new log
I20260806 18:30:14.879357 19008 ts_tablet_manager.cc:1397] T 578df3dca0b349d48291c750c77800b8 P ef0e172af3fb4dc3a3befe1f5068af94: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:30:14.879545 19009 tablet_bootstrap.cc:654] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:14.879781 19008 raft_consensus.cc:348] T 578df3dca0b349d48291c750c77800b8 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.879912 19008 raft_consensus.cc:374] T 578df3dca0b349d48291c750c77800b8 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:14.879959 19008 raft_consensus.cc:729] T 578df3dca0b349d48291c750c77800b8 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef0e172af3fb4dc3a3befe1f5068af94, State: Initialized, Role: FOLLOWER
I20260806 18:30:14.880087 19008 consensus_queue.cc:260] T 578df3dca0b349d48291c750c77800b8 P ef0e172af3fb4dc3a3befe1f5068af94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.880355 19008 ts_tablet_manager.cc:1428] T 578df3dca0b349d48291c750c77800b8 P ef0e172af3fb4dc3a3befe1f5068af94: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:30:14.880365 19010 tablet_bootstrap.cc:492] T 578df3dca0b349d48291c750c77800b8 P 05c9707ab46349aebee2e0952c2574d5: Bootstrap starting.
I20260806 18:30:14.880359 19015 tablet_replica.cc:439] T 578df3dca0b349d48291c750c77800b8 P ef0e172af3fb4dc3a3befe1f5068af94: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } } }
I20260806 18:30:14.880571 19015 tablet_replica.cc:442] T 578df3dca0b349d48291c750c77800b8 P ef0e172af3fb4dc3a3befe1f5068af94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:30:14.881266 19010 tablet_bootstrap.cc:654] T 578df3dca0b349d48291c750c77800b8 P 05c9707ab46349aebee2e0952c2574d5: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:14.881659 19009 tablet_bootstrap.cc:492] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c: No bootstrap required, opened a new log
I20260806 18:30:14.881757 19009 ts_tablet_manager.cc:1397] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 18:30:14.882223 19009 raft_consensus.cc:348] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.882316 19009 raft_consensus.cc:374] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:14.882344 19009 raft_consensus.cc:729] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63537119fb62424fbe5fa9eddce11a4c, State: Initialized, Role: FOLLOWER
I20260806 18:30:14.882481 19010 tablet_bootstrap.cc:492] T 578df3dca0b349d48291c750c77800b8 P 05c9707ab46349aebee2e0952c2574d5: No bootstrap required, opened a new log
I20260806 18:30:14.882458 19009 consensus_queue.cc:260] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.882563 19010 ts_tablet_manager.cc:1397] T 578df3dca0b349d48291c750c77800b8 P 05c9707ab46349aebee2e0952c2574d5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:30:14.882709 19016 tablet_replica.cc:439] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } } }
I20260806 18:30:14.882887 19016 tablet_replica.cc:442] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:30:14.882706 19009 ts_tablet_manager.cc:1428] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:30:14.882982 19010 raft_consensus.cc:348] T 578df3dca0b349d48291c750c77800b8 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.883105 19010 raft_consensus.cc:374] T 578df3dca0b349d48291c750c77800b8 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:14.883147 19010 raft_consensus.cc:729] T 578df3dca0b349d48291c750c77800b8 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05c9707ab46349aebee2e0952c2574d5, State: Initialized, Role: FOLLOWER
I20260806 18:30:14.883271 19010 consensus_queue.cc:260] T 578df3dca0b349d48291c750c77800b8 P 05c9707ab46349aebee2e0952c2574d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.883544 19010 ts_tablet_manager.cc:1428] T 578df3dca0b349d48291c750c77800b8 P 05c9707ab46349aebee2e0952c2574d5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:30:14.884239 19018 tablet_replica.cc:439] T 578df3dca0b349d48291c750c77800b8 P 05c9707ab46349aebee2e0952c2574d5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } } }
I20260806 18:30:14.884371 19018 tablet_replica.cc:442] T 578df3dca0b349d48291c750c77800b8 P 05c9707ab46349aebee2e0952c2574d5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:30:14.900843 19016 raft_consensus.cc:482] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:30:14.900995 19016 raft_consensus.cc:504] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.902202 19016 leader_election.cc:288] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05c9707ab46349aebee2e0952c2574d5 (127.7.67.194:43037), ef0e172af3fb4dc3a3befe1f5068af94 (127.7.67.195:37097)
I20260806 18:30:14.906369 19005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.902544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:45415 (local address 127.7.67.195:37097)
0806 18:30:14.902671 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.902675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.902916 (+ 241us) server_negotiation.cc:407] Connection header received
0806 18:30:14.903074 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.903077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.903128 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.903182 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:14.903497 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.904042 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.905363 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.905806 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.905925 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.906018 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:14.906169 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 18:30:14.906240 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:30:14.906427 19000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.902544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:55991 (local address 127.7.67.194:43037)
0806 18:30:14.902671 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.902675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.903009 (+ 334us) server_negotiation.cc:407] Connection header received
0806 18:30:14.903115 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.903117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.903158 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.903229 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:14.903538 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.904034 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.905439 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.905844 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.905998 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.906059 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:14.906226 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 18:30:14.906303 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:30:14.906451 19061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.902430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:37097 (local address 127.7.67.193:45415)
0806 18:30:14.902808 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:14.902820 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:14.902957 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:14.903210 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:14.903213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:14.903230 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:14.903390 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.903396 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.904152 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.904155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.905239 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.905250 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.905925 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.905932 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.906056 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.906079 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:30:14.906151 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 18:30:14.906222 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":66,"spinlock_wait_cycles":896,"thread_start_us":132,"threads_started":1}
I20260806 18:30:14.906461 19060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.902430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:43037 (local address 127.7.67.193:55991)
0806 18:30:14.902898 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:14.902913 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:14.903027 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:14.903250 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:14.903253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:14.903294 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:14.903421 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.903427 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.904138 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.904141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.905273 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.905285 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.906000 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.906004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.906115 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.906139 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:30:14.906207 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 18:30:14.906275 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":25,"spinlock_wait_cycles":31488,"thread_start_us":116,"threads_started":1}
I20260806 18:30:14.906960 18909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "578df3dca0b349d48291c750c77800b8" candidate_uuid: "63537119fb62424fbe5fa9eddce11a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" is_pre_election: true
I20260806 18:30:14.907125 18909 raft_consensus.cc:2454] T 578df3dca0b349d48291c750c77800b8 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63537119fb62424fbe5fa9eddce11a4c in term 0.
I20260806 18:30:14.907265 18714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "578df3dca0b349d48291c750c77800b8" candidate_uuid: "63537119fb62424fbe5fa9eddce11a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05c9707ab46349aebee2e0952c2574d5" is_pre_election: true
I20260806 18:30:14.907552 18714 raft_consensus.cc:2454] T 578df3dca0b349d48291c750c77800b8 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63537119fb62424fbe5fa9eddce11a4c in term 0.
I20260806 18:30:14.907509 18434 leader_election.cc:302] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63537119fb62424fbe5fa9eddce11a4c, ef0e172af3fb4dc3a3befe1f5068af94; no voters:
I20260806 18:30:14.907804 19016 raft_consensus.cc:2781] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:30:14.907887 19016 raft_consensus.cc:482] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:30:14.907929 19016 raft_consensus.cc:3037] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:14.908963 19016 raft_consensus.cc:504] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.909302 19016 leader_election.cc:288] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c [CANDIDATE]: Term 1 election: Requested vote from peers 05c9707ab46349aebee2e0952c2574d5 (127.7.67.194:43037), ef0e172af3fb4dc3a3befe1f5068af94 (127.7.67.195:37097)
I20260806 18:30:14.909683 18714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "578df3dca0b349d48291c750c77800b8" candidate_uuid: "63537119fb62424fbe5fa9eddce11a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05c9707ab46349aebee2e0952c2574d5"
I20260806 18:30:14.909684 18909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "578df3dca0b349d48291c750c77800b8" candidate_uuid: "63537119fb62424fbe5fa9eddce11a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef0e172af3fb4dc3a3befe1f5068af94"
I20260806 18:30:14.909807 18909 raft_consensus.cc:3037] T 578df3dca0b349d48291c750c77800b8 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:14.909807 18714 raft_consensus.cc:3037] T 578df3dca0b349d48291c750c77800b8 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:14.910959 18909 raft_consensus.cc:2454] T 578df3dca0b349d48291c750c77800b8 P ef0e172af3fb4dc3a3befe1f5068af94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63537119fb62424fbe5fa9eddce11a4c in term 1.
I20260806 18:30:14.911031 18714 raft_consensus.cc:2454] T 578df3dca0b349d48291c750c77800b8 P 05c9707ab46349aebee2e0952c2574d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63537119fb62424fbe5fa9eddce11a4c in term 1.
I20260806 18:30:14.911252 18434 leader_election.cc:302] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63537119fb62424fbe5fa9eddce11a4c, ef0e172af3fb4dc3a3befe1f5068af94; no voters:
I20260806 18:30:14.911432 19016 raft_consensus.cc:2781] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:14.911664 19016 raft_consensus.cc:686] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c [term 1 LEADER]: Becoming Leader. State: Replica: 63537119fb62424fbe5fa9eddce11a4c, State: Running, Role: LEADER
I20260806 18:30:14.911921 19016 consensus_queue.cc:237] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:14.913136 19062 tablet_replica.cc:439] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c: TxnStatusTablet state changed. Reason: New leader 63537119fb62424fbe5fa9eddce11a4c. Latest consensus state: current_term: 1 leader_uuid: "63537119fb62424fbe5fa9eddce11a4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } } }
I20260806 18:30:14.913226 19062 tablet_replica.cc:442] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:30:14.913471 19064 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 18:30:14.913636 19064 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 18:30:14.914419 18312 catalog_manager.cc:5495] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 63537119fb62424fbe5fa9eddce11a4c (127.7.67.193). New cstate: current_term: 1 leader_uuid: "63537119fb62424fbe5fa9eddce11a4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } health_report { overall_health: UNKNOWN } } }
I20260806 18:30:14.934167 19055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.930184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:33997 (local address 127.0.0.1:49942)
0806 18:30:14.930353 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:14.930369 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:14.930494 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:14.930798 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:14.930800 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:14.930806 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:30:14.930929 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.930934 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.931899 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.931903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.932531 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.932540 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.933075 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.933078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.933135 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.933180 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:30:14.933909 (+ 729us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:30:14.933924 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 18:30:14.933979 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 18:30:14.934048 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 18:30:14.934191 19007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.930314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49942 (local address 127.7.67.193:33997)
0806 18:30:14.930464 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.930469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.930486 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:30:14.930630 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.930633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.930698 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.930798 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:30:14.931016 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.931795 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.932633 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.932986 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.933082 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.933297 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:30:14.933821 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:30:14.933931 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:14.933994 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 18:30:14.934055 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:30:14.937062 18909 raft_consensus.cc:1261] T 578df3dca0b349d48291c750c77800b8 P ef0e172af3fb4dc3a3befe1f5068af94 [term 1 FOLLOWER]: Refusing update from remote peer 63537119fb62424fbe5fa9eddce11a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:30:14.937220 18714 raft_consensus.cc:1261] T 578df3dca0b349d48291c750c77800b8 P 05c9707ab46349aebee2e0952c2574d5 [term 1 FOLLOWER]: Refusing update from remote peer 63537119fb62424fbe5fa9eddce11a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:30:14.937613 19062 consensus_queue.cc:1035] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:30:14.937788 19016 consensus_queue.cc:1035] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:30:14.939354 19015 tablet_replica.cc:439] T 578df3dca0b349d48291c750c77800b8 P ef0e172af3fb4dc3a3befe1f5068af94: TxnStatusTablet state changed. Reason: New leader 63537119fb62424fbe5fa9eddce11a4c. Latest consensus state: current_term: 1 leader_uuid: "63537119fb62424fbe5fa9eddce11a4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } } }
I20260806 18:30:14.939476 19015 tablet_replica.cc:442] T 578df3dca0b349d48291c750c77800b8 P ef0e172af3fb4dc3a3befe1f5068af94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:30:14.940901 19062 tablet_replica.cc:439] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "63537119fb62424fbe5fa9eddce11a4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } } }
I20260806 18:30:14.941032 19062 tablet_replica.cc:442] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:30:14.943284 19015 tablet_replica.cc:439] T 578df3dca0b349d48291c750c77800b8 P ef0e172af3fb4dc3a3befe1f5068af94: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "63537119fb62424fbe5fa9eddce11a4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } } }
I20260806 18:30:14.943406 19015 tablet_replica.cc:442] T 578df3dca0b349d48291c750c77800b8 P ef0e172af3fb4dc3a3befe1f5068af94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:30:14.943940 19024 mvcc.cc:205] Tried to move back new op lower bound from 7315623997177024512 to 7315623997081649152. Current Snapshot: MvccSnapshot[applied={T|T < 7315623997177024512 or (T in {7315623997177024512})}]
I20260806 18:30:14.945508 19018 tablet_replica.cc:439] T 578df3dca0b349d48291c750c77800b8 P 05c9707ab46349aebee2e0952c2574d5: TxnStatusTablet state changed. Reason: New leader 63537119fb62424fbe5fa9eddce11a4c. Latest consensus state: current_term: 1 leader_uuid: "63537119fb62424fbe5fa9eddce11a4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } } }
I20260806 18:30:14.945631 19018 tablet_replica.cc:442] T 578df3dca0b349d48291c750c77800b8 P 05c9707ab46349aebee2e0952c2574d5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:30:14.947204 19016 tablet_replica.cc:439] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "63537119fb62424fbe5fa9eddce11a4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } } }
I20260806 18:30:14.947327 19016 tablet_replica.cc:442] T 578df3dca0b349d48291c750c77800b8 P 63537119fb62424fbe5fa9eddce11a4c: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:30:14.948487 19018 tablet_replica.cc:439] T 578df3dca0b349d48291c750c77800b8 P 05c9707ab46349aebee2e0952c2574d5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "63537119fb62424fbe5fa9eddce11a4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } } }
I20260806 18:30:14.948596 19018 tablet_replica.cc:442] T 578df3dca0b349d48291c750c77800b8 P 05c9707ab46349aebee2e0952c2574d5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:30:14.957032 19007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.953503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:46177 (local address 127.7.67.193:33997)
0806 18:30:14.953705 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.953710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.953755 (+ 45us) server_negotiation.cc:407] Connection header received
0806 18:30:14.953864 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.953867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.953916 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.953999 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:30:14.954189 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.954996 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.955808 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.956176 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.956268 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.956552 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:30:14.956637 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:30:14.956698 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:14.956851 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 18:30:14.956927 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260806 18:30:14.957069 19076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.953394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:33997 (local address 127.7.67.194:46177)
0806 18:30:14.953689 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:14.953702 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:14.953796 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:14.954002 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:14.954004 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:14.954010 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:30:14.954103 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.954107 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.955088 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.955091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.955709 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.955717 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.956294 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.956297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.956365 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.956434 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:30:14.956743 (+ 309us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:30:14.956750 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:30:14.956817 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 18:30:14.956904 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":114,"threads_started":1}
I20260806 18:30:14.963927 18532 raft_consensus.cc:1261] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c [term 1 FOLLOWER]: Refusing update from remote peer 05c9707ab46349aebee2e0952c2574d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:30:14.963927 18909 raft_consensus.cc:1261] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94 [term 1 FOLLOWER]: Refusing update from remote peer 05c9707ab46349aebee2e0952c2574d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:30:14.964295 19029 consensus_queue.cc:1035] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:30:14.964457 19018 consensus_queue.cc:1035] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:30:14.987917 19000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.984173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:44977 (local address 127.7.67.194:43037)
0806 18:30:14.984286 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:14.984290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:14.984551 (+ 261us) server_negotiation.cc:407] Connection header received
0806 18:30:14.984669 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:14.984671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:14.984714 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:14.984776 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:30:14.985048 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.985603 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.986743 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.987048 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.987147 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.987494 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:30:14.987571 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:30:14.987628 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:14.987716 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 18:30:14.987772 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:30:14.987941 19080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:14.984100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:43037 (local address 127.7.67.193:44977)
0806 18:30:14.984473 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:14.984489 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:14.984593 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:14.984796 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:14.984798 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:14.984822 (+ 24us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:30:14.984955 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.984961 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.985721 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.985725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.986607 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:14.986618 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:14.987225 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:14.987228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:14.987296 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:14.987342 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:30:14.987644 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:30:14.987651 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:30:14.987708 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 18:30:14.987786 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":100,"threads_started":1}
I20260806 18:30:15.007504 18312 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:30:15.007658 18312 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:30:15.009578 18312 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a: Sending DeleteTablet for 3 replicas of tablet c35ca7f730aa42bfa0c82fec6c20fe67
I20260806 18:30:15.010186 18879 tablet_service.cc:1514] Processing DeleteTablet for tablet c35ca7f730aa42bfa0c82fec6c20fe67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:15 UTC) from {username='oryx', principal='oryx/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:47776
I20260806 18:30:15.010206 18690 tablet_service.cc:1514] Processing DeleteTablet for tablet c35ca7f730aa42bfa0c82fec6c20fe67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:15 UTC) from {username='oryx', principal='oryx/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:45106
I20260806 18:30:15.010368 19036 tablet_replica.cc:330] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94: stopping tablet replica
I20260806 18:30:15.010385 19035 tablet_replica.cc:330] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5: stopping tablet replica
I20260806 18:30:15.010491 19036 raft_consensus.cc:2229] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:30:15.010491 19035 raft_consensus.cc:2229] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:30:15.010604 19036 raft_consensus.cc:2258] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:15.010671 19035 raft_consensus.cc:2258] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:15.010674 18497 tablet_service.cc:1514] Processing DeleteTablet for tablet c35ca7f730aa42bfa0c82fec6c20fe67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:15 UTC) from {username='oryx', principal='oryx/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:49932
I20260806 18:30:15.010874 19038 tablet_replica.cc:330] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c: stopping tablet replica
I20260806 18:30:15.010999 19038 raft_consensus.cc:2229] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:30:15.011112 19038 raft_consensus.cc:2258] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:15.011375 19036 ts_tablet_manager.cc:1905] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:15.011426 19035 ts_tablet_manager.cc:1905] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:15.011834 19038 ts_tablet_manager.cc:1905] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:15.014261 19036 ts_tablet_manager.cc:1918] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:30:15.014336 19036 log.cc:1196] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/c35ca7f730aa42bfa0c82fec6c20fe67
I20260806 18:30:15.014312 19035 ts_tablet_manager.cc:1918] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:30:15.014371 19038 ts_tablet_manager.cc:1918] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:30:15.014389 19035 log.cc:1196] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/c35ca7f730aa42bfa0c82fec6c20fe67
I20260806 18:30:15.014432 19038 log.cc:1196] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/wal/wals/c35ca7f730aa42bfa0c82fec6c20fe67
I20260806 18:30:15.014593 19036 ts_tablet_manager.cc:1939] T c35ca7f730aa42bfa0c82fec6c20fe67 P ef0e172af3fb4dc3a3befe1f5068af94: Deleting consensus metadata
I20260806 18:30:15.014653 19038 ts_tablet_manager.cc:1939] T c35ca7f730aa42bfa0c82fec6c20fe67 P 63537119fb62424fbe5fa9eddce11a4c: Deleting consensus metadata
I20260806 18:30:15.014679 19035 ts_tablet_manager.cc:1939] T c35ca7f730aa42bfa0c82fec6c20fe67 P 05c9707ab46349aebee2e0952c2574d5: Deleting consensus metadata
I20260806 18:30:15.015250 18296 catalog_manager.cc:4841] TS ef0e172af3fb4dc3a3befe1f5068af94 (127.7.67.195:37097): tablet c35ca7f730aa42bfa0c82fec6c20fe67 (table test-table [id=cbaaf06bcc2e4adb8232376e8fbd9da1]) successfully deleted
I20260806 18:30:15.015735 18293 catalog_manager.cc:4841] TS 05c9707ab46349aebee2e0952c2574d5 (127.7.67.194:43037): tablet c35ca7f730aa42bfa0c82fec6c20fe67 (table test-table [id=cbaaf06bcc2e4adb8232376e8fbd9da1]) successfully deleted
I20260806 18:30:15.015847 18293 catalog_manager.cc:4841] TS 63537119fb62424fbe5fa9eddce11a4c (127.7.67.193:33997): tablet c35ca7f730aa42bfa0c82fec6c20fe67 (table test-table [id=cbaaf06bcc2e4adb8232376e8fbd9da1]) successfully deleted
Aug 06 18:30:15 dist-test-slave-5bn2 krb5kdc[18256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:30:15 dist-test-slave-5bn2 krb5kdc[18256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:15 dist-test-slave-5bn2 krb5kdc[18256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:30:15.043534 18381 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:15.035332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34978 (local address 127.7.67.254:33367)
0806 18:30:15.035466 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:15.035470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:15.035775 (+ 305us) server_negotiation.cc:407] Connection header received
0806 18:30:15.035922 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:15.035925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:15.035985 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:15.036064 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:15.036969 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.037750 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.039183 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.039560 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.039657 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:15.043402 (+ 3745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34978: BlockingRecv error: recv got EOF from 127.0.0.1:34978 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 18:30:15 dist-test-slave-5bn2 krb5kdc[18256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM
I20260806 18:30:15.062019 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:15.045854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34988 (local address 127.7.67.254:33367)
0806 18:30:15.045978 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:15.045982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:15.053232 (+ 7250us) server_negotiation.cc:407] Connection header received
0806 18:30:15.053327 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:15.053331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:15.053392 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:15.053494 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:15.054095 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.054906 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.055671 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.056017 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.056133 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:15.058793 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:15.058797 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:15.058800 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:15.058831 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:15.060406 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:15.060801 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:15.060805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:15.060808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:15.060908 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:15.061141 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:15.061144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:15.061145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:15.061383 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:15.061525 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:15.061712 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 18:30:15.061837 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:30:15.064864 18312 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:30:15.065400 18312 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:30:15.070458 18690 tablet_service.cc:1467] Processing CreateTablet for tablet 32495fd36d734ec99818f0850e096ed1 (DEFAULT_TABLE table=test-table [id=f98ef6d712db4dc1ace5543ef3dfd14e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:15.070458 18497 tablet_service.cc:1467] Processing CreateTablet for tablet 32495fd36d734ec99818f0850e096ed1 (DEFAULT_TABLE table=test-table [id=f98ef6d712db4dc1ace5543ef3dfd14e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:15.070763 18497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32495fd36d734ec99818f0850e096ed1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:15.070765 18690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32495fd36d734ec99818f0850e096ed1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:15.072989 19010 tablet_bootstrap.cc:492] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5: Bootstrap starting.
I20260806 18:30:15.073143 19009 tablet_bootstrap.cc:492] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c: Bootstrap starting.
I20260806 18:30:15.073575 18879 tablet_service.cc:1467] Processing CreateTablet for tablet 32495fd36d734ec99818f0850e096ed1 (DEFAULT_TABLE table=test-table [id=f98ef6d712db4dc1ace5543ef3dfd14e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:15.073839 18879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32495fd36d734ec99818f0850e096ed1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:15.073951 19010 tablet_bootstrap.cc:654] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:15.074482 19009 tablet_bootstrap.cc:654] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:15.075618 19010 tablet_bootstrap.cc:492] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5: No bootstrap required, opened a new log
I20260806 18:30:15.075704 19010 ts_tablet_manager.cc:1397] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 18:30:15.076062 19010 raft_consensus.cc:348] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:15.076200 19010 raft_consensus.cc:374] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:15.076227 19008 tablet_bootstrap.cc:492] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94: Bootstrap starting.
I20260806 18:30:15.076236 19010 raft_consensus.cc:729] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05c9707ab46349aebee2e0952c2574d5, State: Initialized, Role: FOLLOWER
I20260806 18:30:15.076359 19010 consensus_queue.cc:260] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:15.076632 19010 ts_tablet_manager.cc:1428] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:30:15.077173 19008 tablet_bootstrap.cc:654] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:15.078066 19009 tablet_bootstrap.cc:492] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c: No bootstrap required, opened a new log
I20260806 18:30:15.078148 19009 ts_tablet_manager.cc:1397] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 18:30:15.078645 19009 raft_consensus.cc:348] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:15.078765 19009 raft_consensus.cc:374] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:15.078797 19009 raft_consensus.cc:729] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63537119fb62424fbe5fa9eddce11a4c, State: Initialized, Role: FOLLOWER
I20260806 18:30:15.078833 19008 tablet_bootstrap.cc:492] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94: No bootstrap required, opened a new log
I20260806 18:30:15.078945 19008 ts_tablet_manager.cc:1397] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:30:15.079128 19009 consensus_queue.cc:260] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:15.079382 19008 raft_consensus.cc:348] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:15.079486 19008 raft_consensus.cc:374] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:15.079516 19008 raft_consensus.cc:729] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef0e172af3fb4dc3a3befe1f5068af94, State: Initialized, Role: FOLLOWER
I20260806 18:30:15.079633 19008 consensus_queue.cc:260] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:15.079797 19009 ts_tablet_manager.cc:1428] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:30:15.079878 19008 ts_tablet_manager.cc:1428] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:30:15.110177 19020 raft_consensus.cc:482] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:30:15.110322 19020 raft_consensus.cc:504] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:15.110745 19020 leader_election.cc:288] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63537119fb62424fbe5fa9eddce11a4c (127.7.67.193:33997), ef0e172af3fb4dc3a3befe1f5068af94 (127.7.67.195:37097)
I20260806 18:30:15.111183 18532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32495fd36d734ec99818f0850e096ed1" candidate_uuid: "05c9707ab46349aebee2e0952c2574d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63537119fb62424fbe5fa9eddce11a4c" is_pre_election: true
I20260806 18:30:15.111197 18909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32495fd36d734ec99818f0850e096ed1" candidate_uuid: "05c9707ab46349aebee2e0952c2574d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" is_pre_election: true
I20260806 18:30:15.111322 18532 raft_consensus.cc:2454] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05c9707ab46349aebee2e0952c2574d5 in term 0.
I20260806 18:30:15.111361 18909 raft_consensus.cc:2454] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05c9707ab46349aebee2e0952c2574d5 in term 0.
I20260806 18:30:15.111668 18629 leader_election.cc:302] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05c9707ab46349aebee2e0952c2574d5, ef0e172af3fb4dc3a3befe1f5068af94; no voters:
I20260806 18:30:15.111851 19020 raft_consensus.cc:2781] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:30:15.111925 19020 raft_consensus.cc:482] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:30:15.111968 19020 raft_consensus.cc:3037] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:15.112987 19020 raft_consensus.cc:504] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:15.113318 19020 leader_election.cc:288] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [CANDIDATE]: Term 1 election: Requested vote from peers 63537119fb62424fbe5fa9eddce11a4c (127.7.67.193:33997), ef0e172af3fb4dc3a3befe1f5068af94 (127.7.67.195:37097)
I20260806 18:30:15.113662 18532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32495fd36d734ec99818f0850e096ed1" candidate_uuid: "05c9707ab46349aebee2e0952c2574d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63537119fb62424fbe5fa9eddce11a4c"
I20260806 18:30:15.113682 18909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32495fd36d734ec99818f0850e096ed1" candidate_uuid: "05c9707ab46349aebee2e0952c2574d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef0e172af3fb4dc3a3befe1f5068af94"
I20260806 18:30:15.113780 18909 raft_consensus.cc:3037] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:15.113788 18532 raft_consensus.cc:3037] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:15.114665 18532 raft_consensus.cc:2454] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05c9707ab46349aebee2e0952c2574d5 in term 1.
I20260806 18:30:15.114673 18909 raft_consensus.cc:2454] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05c9707ab46349aebee2e0952c2574d5 in term 1.
I20260806 18:30:15.114943 18629 leader_election.cc:302] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05c9707ab46349aebee2e0952c2574d5, ef0e172af3fb4dc3a3befe1f5068af94; no voters:
I20260806 18:30:15.115162 19020 raft_consensus.cc:2781] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:15.115270 19020 raft_consensus.cc:686] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [term 1 LEADER]: Becoming Leader. State: Replica: 05c9707ab46349aebee2e0952c2574d5, State: Running, Role: LEADER
I20260806 18:30:15.115411 19020 consensus_queue.cc:237] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:15.117038 18312 catalog_manager.cc:5495] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05c9707ab46349aebee2e0952c2574d5 (127.7.67.194). New cstate: current_term: 1 leader_uuid: "05c9707ab46349aebee2e0952c2574d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } health_report { overall_health: UNKNOWN } } }
I20260806 18:30:15.134181 19000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:15.131568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45128 (local address 127.7.67.194:43037)
0806 18:30:15.131693 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:15.131696 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:15.131710 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:30:15.131798 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:15.131800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:15.131848 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:15.131910 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:30:15.132115 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.132620 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.133367 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.133575 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.133641 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:15.133826 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:30:15.133890 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:30:15.133935 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:15.134000 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 18:30:15.134044 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:30:15.136143 18909 raft_consensus.cc:1261] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94 [term 1 FOLLOWER]: Refusing update from remote peer 05c9707ab46349aebee2e0952c2574d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:30:15.136142 18532 raft_consensus.cc:1261] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c [term 1 FOLLOWER]: Refusing update from remote peer 05c9707ab46349aebee2e0952c2574d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:30:15.136552 19029 consensus_queue.cc:1035] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:30:15.136719 19020 consensus_queue.cc:1035] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:30:15.143383 18315 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:30:15.143548 18315 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:30:15.145706 18315 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a: Sending DeleteTablet for 3 replicas of tablet 32495fd36d734ec99818f0850e096ed1
I20260806 18:30:15.146251 18497 tablet_service.cc:1514] Processing DeleteTablet for tablet 32495fd36d734ec99818f0850e096ed1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:15 UTC) from {username='oryx', principal='oryx/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:49932
I20260806 18:30:15.146450 18690 tablet_service.cc:1514] Processing DeleteTablet for tablet 32495fd36d734ec99818f0850e096ed1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:15 UTC) from {username='oryx', principal='oryx/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:45106
I20260806 18:30:15.146550 19038 tablet_replica.cc:330] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c: stopping tablet replica
I20260806 18:30:15.146577 18878 tablet_service.cc:1514] Processing DeleteTablet for tablet 32495fd36d734ec99818f0850e096ed1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:15 UTC) from {username='oryx', principal='oryx/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:47776
I20260806 18:30:15.146616 19035 tablet_replica.cc:330] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5: stopping tablet replica
I20260806 18:30:15.146687 19038 raft_consensus.cc:2229] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:30:15.146725 19035 raft_consensus.cc:2229] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:30:15.146754 19036 tablet_replica.cc:330] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94: stopping tablet replica
I20260806 18:30:15.146823 19038 raft_consensus.cc:2258] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:15.146891 19036 raft_consensus.cc:2229] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:30:15.146924 19035 raft_consensus.cc:2258] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:15.147017 19036 raft_consensus.cc:2258] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:15.147013 18879 tablet_service.cc:1514] Processing DeleteTablet for tablet 32495fd36d734ec99818f0850e096ed1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:15 UTC) from {username='oryx', principal='oryx/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:47776
I20260806 18:30:15.147579 19035 ts_tablet_manager.cc:1905] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:15.147678 19036 ts_tablet_manager.cc:1905] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:15.147704 18315 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 18:30:15.147892 19038 ts_tablet_manager.cc:1905] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 18:30:15.148100 18315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:30:15.149936 19038 ts_tablet_manager.cc:1918] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:30:15.149992 19038 log.cc:1196] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-0/wal/wals/32495fd36d734ec99818f0850e096ed1
I20260806 18:30:15.150003 19035 ts_tablet_manager.cc:1918] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:30:15.150003 19036 ts_tablet_manager.cc:1918] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:30:15.150091 19035 log.cc:1196] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/32495fd36d734ec99818f0850e096ed1
I20260806 18:30:15.150091 19036 log.cc:1196] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/32495fd36d734ec99818f0850e096ed1
I20260806 18:30:15.150182 19038 ts_tablet_manager.cc:1939] T 32495fd36d734ec99818f0850e096ed1 P 63537119fb62424fbe5fa9eddce11a4c: Deleting consensus metadata
I20260806 18:30:15.150339 19035 ts_tablet_manager.cc:1939] T 32495fd36d734ec99818f0850e096ed1 P 05c9707ab46349aebee2e0952c2574d5: Deleting consensus metadata
I20260806 18:30:15.150388 19036 ts_tablet_manager.cc:1939] T 32495fd36d734ec99818f0850e096ed1 P ef0e172af3fb4dc3a3befe1f5068af94: Deleting consensus metadata
I20260806 18:30:15.150892 18293 catalog_manager.cc:4841] TS 63537119fb62424fbe5fa9eddce11a4c (127.7.67.193:33997): tablet 32495fd36d734ec99818f0850e096ed1 (table test-table [id=f98ef6d712db4dc1ace5543ef3dfd14e]) successfully deleted
I20260806 18:30:15.151052 18293 catalog_manager.cc:4841] TS 05c9707ab46349aebee2e0952c2574d5 (127.7.67.194:43037): tablet 32495fd36d734ec99818f0850e096ed1 (table test-table [id=f98ef6d712db4dc1ace5543ef3dfd14e]) successfully deleted
I20260806 18:30:15.151114 18296 catalog_manager.cc:4841] TS ef0e172af3fb4dc3a3befe1f5068af94 (127.7.67.195:37097): tablet 32495fd36d734ec99818f0850e096ed1 (table test-table [id=f98ef6d712db4dc1ace5543ef3dfd14e]) successfully deleted
W20260806 18:30:15.151400 18296 catalog_manager.cc:4808] TS ef0e172af3fb4dc3a3befe1f5068af94 (127.7.67.195:37097): delete failed for tablet 32495fd36d734ec99818f0850e096ed1 because the tablet was not found. No further retry: Not found: Tablet not found: 32495fd36d734ec99818f0850e096ed1
I20260806 18:30:15.154944 18690 tablet_service.cc:1467] Processing CreateTablet for tablet bc78611d40d249349520a49ea50f168c (DEFAULT_TABLE table=test-table [id=20b8c89b491b485cba35b8416089305b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:15.154939 18497 tablet_service.cc:1467] Processing CreateTablet for tablet bc78611d40d249349520a49ea50f168c (DEFAULT_TABLE table=test-table [id=20b8c89b491b485cba35b8416089305b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:15.155084 18879 tablet_service.cc:1467] Processing CreateTablet for tablet bc78611d40d249349520a49ea50f168c (DEFAULT_TABLE table=test-table [id=20b8c89b491b485cba35b8416089305b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:15.155201 18690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc78611d40d249349520a49ea50f168c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:15.155200 18497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc78611d40d249349520a49ea50f168c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:15.155347 18879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc78611d40d249349520a49ea50f168c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:15.157296 19008 tablet_bootstrap.cc:492] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94: Bootstrap starting.
I20260806 18:30:15.157423 19009 tablet_bootstrap.cc:492] T bc78611d40d249349520a49ea50f168c P 63537119fb62424fbe5fa9eddce11a4c: Bootstrap starting.
I20260806 18:30:15.157428 19010 tablet_bootstrap.cc:492] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5: Bootstrap starting.
I20260806 18:30:15.158195 19008 tablet_bootstrap.cc:654] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:15.158247 19010 tablet_bootstrap.cc:654] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:15.158246 19009 tablet_bootstrap.cc:654] T bc78611d40d249349520a49ea50f168c P 63537119fb62424fbe5fa9eddce11a4c: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:15.159436 19010 tablet_bootstrap.cc:492] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5: No bootstrap required, opened a new log
I20260806 18:30:15.159523 19010 ts_tablet_manager.cc:1397] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:30:15.159644 19008 tablet_bootstrap.cc:492] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94: No bootstrap required, opened a new log
I20260806 18:30:15.159683 19009 tablet_bootstrap.cc:492] T bc78611d40d249349520a49ea50f168c P 63537119fb62424fbe5fa9eddce11a4c: No bootstrap required, opened a new log
I20260806 18:30:15.159722 19008 ts_tablet_manager.cc:1397] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 18:30:15.159749 19009 ts_tablet_manager.cc:1397] T bc78611d40d249349520a49ea50f168c P 63537119fb62424fbe5fa9eddce11a4c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:30:15.159830 19010 raft_consensus.cc:348] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:15.159912 19010 raft_consensus.cc:374] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:15.159931 19010 raft_consensus.cc:729] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05c9707ab46349aebee2e0952c2574d5, State: Initialized, Role: FOLLOWER
I20260806 18:30:15.160032 19010 consensus_queue.cc:260] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:15.160130 19008 raft_consensus.cc:348] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:15.160130 19009 raft_consensus.cc:348] T bc78611d40d249349520a49ea50f168c P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:15.160235 19009 raft_consensus.cc:374] T bc78611d40d249349520a49ea50f168c P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:15.160233 19008 raft_consensus.cc:374] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:15.160265 19008 raft_consensus.cc:729] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef0e172af3fb4dc3a3befe1f5068af94, State: Initialized, Role: FOLLOWER
I20260806 18:30:15.160265 19009 raft_consensus.cc:729] T bc78611d40d249349520a49ea50f168c P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63537119fb62424fbe5fa9eddce11a4c, State: Initialized, Role: FOLLOWER
I20260806 18:30:15.160269 19010 ts_tablet_manager.cc:1428] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:30:15.160382 19008 consensus_queue.cc:260] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:15.160382 19009 consensus_queue.cc:260] T bc78611d40d249349520a49ea50f168c P 63537119fb62424fbe5fa9eddce11a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:15.160636 19008 ts_tablet_manager.cc:1428] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:30:15.160638 19009 ts_tablet_manager.cc:1428] T bc78611d40d249349520a49ea50f168c P 63537119fb62424fbe5fa9eddce11a4c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:30:15.284126 19015 raft_consensus.cc:482] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:30:15.284283 19015 raft_consensus.cc:504] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:15.285434 19015 leader_election.cc:288] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63537119fb62424fbe5fa9eddce11a4c (127.7.67.193:33997), 05c9707ab46349aebee2e0952c2574d5 (127.7.67.194:43037)
I20260806 18:30:15.289708 19000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:15.286252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:33015 (local address 127.7.67.194:43037)
0806 18:30:15.286388 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:15.286392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:15.286506 (+ 114us) server_negotiation.cc:407] Connection header received
0806 18:30:15.286613 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:15.286616 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:15.286659 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:15.286730 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:15.287105 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.287606 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.288713 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.289104 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.289216 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:15.289270 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:15.289499 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 18:30:15.289575 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:30:15.289727 19007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:15.285788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:59417 (local address 127.7.67.193:33997)
0806 18:30:15.285957 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:15.285962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:15.286100 (+ 138us) server_negotiation.cc:407] Connection header received
0806 18:30:15.286330 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:15.286334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:15.286385 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:15.286448 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:15.286902 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.287514 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.288650 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.289077 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.289183 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:15.289241 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:15.289547 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 18:30:15.289624 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:30:15.289733 19116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:15.285698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:33997 (local address 127.7.67.195:59417)
0806 18:30:15.286016 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:15.286030 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:15.286187 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:15.286611 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:15.286614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:15.286636 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:15.286783 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:15.286791 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.287623 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.287627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:15.288533 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:15.288543 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.289220 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.289224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:15.289389 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:15.289433 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 18:30:15.289506 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 18:30:15.289575 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":39,"thread_start_us":90,"threads_started":1}
I20260806 18:30:15.289736 19118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:15.286115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:43037 (local address 127.7.67.195:33015)
0806 18:30:15.286416 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:15.286429 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:15.286526 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:15.286811 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:15.286815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:15.286835 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:15.286986 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:15.286993 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.287734 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.287737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:15.288584 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:15.288594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.289221 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.289225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:15.289380 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:15.289401 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:30:15.289460 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 18:30:15.289525 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":35,"thread_start_us":63,"threads_started":1}
I20260806 18:30:15.290338 18714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc78611d40d249349520a49ea50f168c" candidate_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05c9707ab46349aebee2e0952c2574d5" is_pre_election: true
I20260806 18:30:15.290342 18532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc78611d40d249349520a49ea50f168c" candidate_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63537119fb62424fbe5fa9eddce11a4c" is_pre_election: true
I20260806 18:30:15.290488 18532 raft_consensus.cc:2454] T bc78611d40d249349520a49ea50f168c P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef0e172af3fb4dc3a3befe1f5068af94 in term 0.
I20260806 18:30:15.290491 18714 raft_consensus.cc:2454] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef0e172af3fb4dc3a3befe1f5068af94 in term 0.
I20260806 18:30:15.290887 18806 leader_election.cc:302] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63537119fb62424fbe5fa9eddce11a4c, ef0e172af3fb4dc3a3befe1f5068af94; no voters:
I20260806 18:30:15.291168 19015 raft_consensus.cc:2781] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:30:15.291227 19015 raft_consensus.cc:482] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:30:15.291270 19015 raft_consensus.cc:3037] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:15.292282 19015 raft_consensus.cc:504] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:15.292613 19015 leader_election.cc:288] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [CANDIDATE]: Term 1 election: Requested vote from peers 63537119fb62424fbe5fa9eddce11a4c (127.7.67.193:33997), 05c9707ab46349aebee2e0952c2574d5 (127.7.67.194:43037)
I20260806 18:30:15.292984 18532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc78611d40d249349520a49ea50f168c" candidate_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63537119fb62424fbe5fa9eddce11a4c"
I20260806 18:30:15.292990 18714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc78611d40d249349520a49ea50f168c" candidate_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05c9707ab46349aebee2e0952c2574d5"
I20260806 18:30:15.293095 18532 raft_consensus.cc:3037] T bc78611d40d249349520a49ea50f168c P 63537119fb62424fbe5fa9eddce11a4c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:15.293099 18714 raft_consensus.cc:3037] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:15.294000 18532 raft_consensus.cc:2454] T bc78611d40d249349520a49ea50f168c P 63537119fb62424fbe5fa9eddce11a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef0e172af3fb4dc3a3befe1f5068af94 in term 1.
I20260806 18:30:15.294016 18714 raft_consensus.cc:2454] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef0e172af3fb4dc3a3befe1f5068af94 in term 1.
I20260806 18:30:15.294270 18806 leader_election.cc:302] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05c9707ab46349aebee2e0952c2574d5, ef0e172af3fb4dc3a3befe1f5068af94; no voters:
I20260806 18:30:15.294445 19015 raft_consensus.cc:2781] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:15.294629 19015 raft_consensus.cc:686] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [term 1 LEADER]: Becoming Leader. State: Replica: ef0e172af3fb4dc3a3befe1f5068af94, State: Running, Role: LEADER
I20260806 18:30:15.294919 19015 consensus_queue.cc:237] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } }
I20260806 18:30:15.297185 18315 catalog_manager.cc:5495] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef0e172af3fb4dc3a3befe1f5068af94 (127.7.67.195). New cstate: current_term: 1 leader_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef0e172af3fb4dc3a3befe1f5068af94" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37097 } health_report { overall_health: HEALTHY } } }
I20260806 18:30:15.321034 19005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:15.317573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47792 (local address 127.7.67.195:37097)
0806 18:30:15.317688 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:15.317692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:15.317706 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:30:15.317860 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:15.317862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:15.317916 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:15.318019 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:30:15.318235 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.318728 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.319555 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.319810 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.319898 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:15.320156 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:30:15.320645 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:30:15.320745 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:15.320835 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 18:30:15.320886 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:30:15.330780 19007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:15.327705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:55163 (local address 127.7.67.193:33997)
0806 18:30:15.327832 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:15.327836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:15.327933 (+ 97us) server_negotiation.cc:407] Connection header received
0806 18:30:15.328078 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:15.328080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:15.328130 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:15.328192 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:30:15.328433 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.328927 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.329766 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.329977 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.330063 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:15.330359 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:30:15.330431 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:30:15.330496 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:15.330579 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 18:30:15.330630 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:30:15.330813 19123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:15.327564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:33997 (local address 127.7.67.195:55163)
0806 18:30:15.327853 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:15.327868 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:15.327971 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:15.328207 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:15.328210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:15.328218 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:30:15.328338 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:15.328343 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.329029 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.329032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:15.329654 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:15.329662 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.330086 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.330090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:15.330189 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:15.330233 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:30:15.330505 (+ 272us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:30:15.330511 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:30:15.330571 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 18:30:15.330650 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":104,"threads_started":1}
I20260806 18:30:15.336380 18714 raft_consensus.cc:1261] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5 [term 1 FOLLOWER]: Refusing update from remote peer ef0e172af3fb4dc3a3befe1f5068af94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:30:15.336436 18532 raft_consensus.cc:1261] T bc78611d40d249349520a49ea50f168c P 63537119fb62424fbe5fa9eddce11a4c [term 1 FOLLOWER]: Refusing update from remote peer ef0e172af3fb4dc3a3befe1f5068af94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:30:15.336826 19119 consensus_queue.cc:1035] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05c9707ab46349aebee2e0952c2574d5" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:30:15.337007 19015 consensus_queue.cc:1035] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63537119fb62424fbe5fa9eddce11a4c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:30:15.353771 19080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:15.350654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:37097 (local address 127.7.67.193:56317)
0806 18:30:15.350785 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:15.350800 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:15.350901 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:15.351244 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:15.351246 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:15.351254 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:30:15.351347 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:15.351351 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.352172 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.352175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:15.352758 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:15.352767 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.353142 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.353144 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:15.353211 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:15.353244 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:30:15.353511 (+ 267us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:30:15.353518 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:30:15.353573 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 18:30:15.353663 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 18:30:15.353775 19005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:15.350720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:56317 (local address 127.7.67.195:37097)
0806 18:30:15.351057 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:15.351061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:15.351073 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:30:15.351110 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:15.351113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:15.351151 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:15.351218 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:30:15.351452 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.352078 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.352845 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:15.353069 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:15.353137 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:15.353346 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:30:15.353427 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:30:15.353487 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:15.353587 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 18:30:15.353641 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251}
I20260806 18:30:15.372722 18312 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:30:15.372881 18312 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:30:15.375056 18312 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 456b0ab10b6d40bb97f0e42f1c3baa1a: Sending DeleteTablet for 3 replicas of tablet bc78611d40d249349520a49ea50f168c
I20260806 18:30:15.375654 18879 tablet_service.cc:1514] Processing DeleteTablet for tablet bc78611d40d249349520a49ea50f168c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:15 UTC) from {username='oryx', principal='oryx/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:47776
I20260806 18:30:15.375725 18497 tablet_service.cc:1514] Processing DeleteTablet for tablet bc78611d40d249349520a49ea50f168c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:15 UTC) from {username='oryx', principal='oryx/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:49932
I20260806 18:30:15.375842 19036 tablet_replica.cc:330] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94: stopping tablet replica
I20260806 18:30:15.375931 19038 tablet_replica.cc:330] T bc78611d40d249349520a49ea50f168c P 63537119fb62424fbe5fa9eddce11a4c: stopping tablet replica
I20260806 18:30:15.375952 19036 raft_consensus.cc:2229] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:30:15.376040 19038 raft_consensus.cc:2229] T bc78611d40d249349520a49ea50f168c P 63537119fb62424fbe5fa9eddce11a4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:30:15.376183 19038 raft_consensus.cc:2258] T bc78611d40d249349520a49ea50f168c P 63537119fb62424fbe5fa9eddce11a4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:15.376266 19036 raft_consensus.cc:2258] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:15.376412 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 18410
I20260806 18:30:15.380081 18690 tablet_service.cc:1514] Processing DeleteTablet for tablet bc78611d40d249349520a49ea50f168c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:15 UTC) from {username='oryx', principal='oryx/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:45106
I20260806 18:30:15.384816 19035 tablet_replica.cc:330] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5: stopping tablet replica
I20260806 18:30:15.384972 19035 raft_consensus.cc:2229] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:30:15.385111 19035 raft_consensus.cc:2258] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:15.385087 19036 ts_tablet_manager.cc:1905] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:15.388116 19035 ts_tablet_manager.cc:1905] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:15.390718 19035 ts_tablet_manager.cc:1918] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:30:15.390813 19035 log.cc:1196] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/bc78611d40d249349520a49ea50f168c
I20260806 18:30:15.391094 19035 ts_tablet_manager.cc:1939] T bc78611d40d249349520a49ea50f168c P 05c9707ab46349aebee2e0952c2574d5: Deleting consensus metadata
I20260806 18:30:15.391911 18293 catalog_manager.cc:4841] TS 05c9707ab46349aebee2e0952c2574d5 (127.7.67.194:43037): tablet bc78611d40d249349520a49ea50f168c (table test-table [id=20b8c89b491b485cba35b8416089305b]) successfully deleted
I20260806 18:30:15.392198 19036 ts_tablet_manager.cc:1918] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:30:15.392284 19036 log.cc:1196] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/bc78611d40d249349520a49ea50f168c
I20260806 18:30:15.392539 19036 ts_tablet_manager.cc:1939] T bc78611d40d249349520a49ea50f168c P ef0e172af3fb4dc3a3befe1f5068af94: Deleting consensus metadata
I20260806 18:30:15.393302 18296 catalog_manager.cc:4841] TS ef0e172af3fb4dc3a3befe1f5068af94 (127.7.67.195:37097): tablet bc78611d40d249349520a49ea50f168c (table test-table [id=20b8c89b491b485cba35b8416089305b]) successfully deleted
W20260806 18:30:15.397851 18293 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.193:33997 (error 108)
I20260806 18:30:15.398030 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 18603
I20260806 18:30:15.403502 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:15.402272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:33997 (local address 127.0.0.1:49946)
0806 18:30:15.403300 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:15.403386 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:33997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":935,"thread_start_us":853,"threads_started":1}
W20260806 18:30:15.405941 18293 catalog_manager.cc:4584] TS 63537119fb62424fbe5fa9eddce11a4c (127.7.67.193:33997): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bc78611d40d249349520a49ea50f168c: Network error: Client connection negotiation failed: client connection to 127.7.67.193:33997: connect: Connection refused (error 111)
I20260806 18:30:15.413251 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 18784
I20260806 18:30:15.425918 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 18271
2026-08-06T18:30:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3116 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:30:15 dist-test-slave-5bn2 krb5kdc[19135](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:30:15 dist-test-slave-5bn2 krb5kdc[19135](info): set up 2 sockets
Aug 06 18:30:15 dist-test-slave-5bn2 krb5kdc[19135](info): commencing operation
krb5kdc: starting...
W20260806 18:30:16.544576 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:30:16 dist-test-slave-5bn2 krb5kdc[19135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:30:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:30:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:30:16.671885 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:41111
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:41111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:16.786043 19150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:16.786362 19150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:16.786423 19150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:16.790481 19150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:30:16.790553 19150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:16.790580 19150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:16.790601 19150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:30:16.790621 19150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:30:16.796100 19150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:41111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:41111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:16.797245 19150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:16.798305 19150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:16.807588 19158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:16.807694 19150 server_base.cc:1034] running on GCE node
W20260806 18:30:16.807593 19155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:16.807588 19156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:16.808508 19150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:16.809648 19150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:16.810812 19150 hybrid_clock.cc:648] HybridClock initialized: now 1786041016810798 us; error 26 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T18:30:16Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1387 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:30:16 dist-test-slave-5bn2 krb5kdc[19163](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:30:16 dist-test-slave-5bn2 krb5kdc[19163](info): set up 2 sockets
Aug 06 18:30:16 dist-test-slave-5bn2 krb5kdc[19163](info): commencing operation
krb5kdc: starting...
W20260806 18:30:17.939841 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:30:18 dist-test-slave-5bn2 krb5kdc[19163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:30:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:30:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:30:18.065548 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34483
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:41565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:34483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:18.177274 19181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:18.177543 19181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:18.177598 19181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:18.181547 19181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:30:18.181607 19181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:18.181632 19181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:18.181653 19181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:30:18.181677 19181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:30:18.187016 19181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:41565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:34483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:18.188056 19181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:18.189069 19181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:18.197216 19189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:18.197252 19187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:18.197245 19193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:18.197468 19181 server_base.cc:1034] running on GCE node
I20260806 18:30:18.198079 19181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:18.199179 19181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:18.200333 19181 hybrid_clock.cc:648] HybridClock initialized: now 1786041018200319 us; error 25 us; skew 500 ppm
Aug 06 18:30:18 dist-test-slave-5bn2 krb5kdc[19163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041018, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:18.203065 19181 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:30:18.203861 19181 webserver.cc:460] Webserver started at http://127.7.67.254:40275/ using document root <none> and password file <none>
I20260806 18:30:18.204380 19181 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:18.204444 19181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:18.204609 19181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:18.206574 19181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "ed34b4cd1a68472f80d938d1ec5298ff"
format_stamp: "Formatted at 2026-08-06 18:30:18 on dist-test-slave-5bn2"
server_key: "3d11e8d1dea30639bb06cc8a29e00d74"
server_key_iv: "9bc64420d1d294bba1309bf1d5685b59"
server_key_version: "encryptionkey@0"
I20260806 18:30:18.207129 19181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "ed34b4cd1a68472f80d938d1ec5298ff"
format_stamp: "Formatted at 2026-08-06 18:30:18 on dist-test-slave-5bn2"
server_key: "3d11e8d1dea30639bb06cc8a29e00d74"
server_key_iv: "9bc64420d1d294bba1309bf1d5685b59"
server_key_version: "encryptionkey@0"
I20260806 18:30:18.210701 19181 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:30:18.213080 19204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:18.214253 19181 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:30:18.214361 19181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "ed34b4cd1a68472f80d938d1ec5298ff"
format_stamp: "Formatted at 2026-08-06 18:30:18 on dist-test-slave-5bn2"
server_key: "3d11e8d1dea30639bb06cc8a29e00d74"
server_key_iv: "9bc64420d1d294bba1309bf1d5685b59"
server_key_version: "encryptionkey@0"
I20260806 18:30:18.214475 19181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:18.228569 19181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:18.231657 19181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:18.231837 19181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:18.238759 19181 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:34483
I20260806 18:30:18.238816 19277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:34483 every 8 connection(s)
I20260806 18:30:18.239744 19181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:30:18.242421 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 19181
I20260806 18:30:18.242571 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:30:18.242620 19279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:18.242933 7439 external_mini_cluster.cc:1428] Setting key 173bc2fbf4892c13912ce6a003ca275e
I20260806 18:30:18.249691 19279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff: Bootstrap starting.
Aug 06 18:30:18 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:30:18.252457 19279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:18.253321 19279 log.cc:824] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:18.255505 19279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff: No bootstrap required, opened a new log
I20260806 18:30:18.256861 19285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:18.244334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55710 (local address 127.7.67.254:34483)
0806 18:30:18.244733 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:18.244748 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:18.244784 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:30:18.245420 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:18.245437 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:18.245755 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:18.246024 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:18.246939 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:18.247935 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:18.249075 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:18.249524 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:18.249633 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:18.252375 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:18.252380 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:18.252416 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:18.252445 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:18.254096 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:18.254565 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:18.254569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:18.254574 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:18.254683 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:18.254980 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:18.254982 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:18.254983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:18.255371 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:18.255536 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:18.255828 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 18:30:18.256113 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":113,"threads_started":1}
I20260806 18:30:18.259771 19279 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed34b4cd1a68472f80d938d1ec5298ff" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34483 } }
I20260806 18:30:18.260031 19279 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:18.260092 19279 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed34b4cd1a68472f80d938d1ec5298ff, State: Initialized, Role: FOLLOWER
I20260806 18:30:18.260496 19279 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed34b4cd1a68472f80d938d1ec5298ff" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34483 } }
I20260806 18:30:18.260623 19279 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:30:18.260689 19279 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:30:18.260787 19279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:18.261710 19279 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed34b4cd1a68472f80d938d1ec5298ff" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34483 } }
I20260806 18:30:18.262027 19279 leader_election.cc:302] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed34b4cd1a68472f80d938d1ec5298ff; no voters:
I20260806 18:30:18.262351 19279 leader_election.cc:288] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:30:18.262449 19287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:18.262733 19287 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [term 1 LEADER]: Becoming Leader. State: Replica: ed34b4cd1a68472f80d938d1ec5298ff, State: Running, Role: LEADER
I20260806 18:30:18.263072 19287 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed34b4cd1a68472f80d938d1ec5298ff" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34483 } }
I20260806 18:30:18.263684 19279 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:30:18.264747 19288 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed34b4cd1a68472f80d938d1ec5298ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed34b4cd1a68472f80d938d1ec5298ff" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34483 } } }
I20260806 18:30:18.264885 19288 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:18.265143 19289 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed34b4cd1a68472f80d938d1ec5298ff. Latest consensus state: current_term: 1 leader_uuid: "ed34b4cd1a68472f80d938d1ec5298ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed34b4cd1a68472f80d938d1ec5298ff" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34483 } } }
I20260806 18:30:18.265234 19289 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:18.265347 19296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:30:18.268457 19296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:30:18.274111 19296 catalog_manager.cc:1347] Generated new cluster ID: 4e2cae676bba4865beb03485eb810f74
I20260806 18:30:18.274180 19296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:30:18.293776 19296 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:30:18.294729 19296 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:30:18.299754 19296 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff: Generated new TSK 0
I20260806 18:30:18.300558 19296 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
I20260806 18:30:18.337869 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:34483
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:41565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:18.455655 19314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:18.455914 19314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:18.455956 19314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:18.459916 19314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:18.459976 19314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:18.460043 19314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:30:18.465477 19314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:41565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:34483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:18.466558 19314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:18.467664 19314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:18.477289 19320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:18.477312 19321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:18.477289 19323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:18.477638 19314 server_base.cc:1034] running on GCE node
I20260806 18:30:18.478061 19314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:18.478688 19314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:18.479851 19314 hybrid_clock.cc:648] HybridClock initialized: now 1786041018479827 us; error 38 us; skew 500 ppm
Aug 06 18:30:18 dist-test-slave-5bn2 krb5kdc[19163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041018, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:18.482537 19314 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:30:18.483566 19314 webserver.cc:460] Webserver started at http://127.7.67.193:43157/ using document root <none> and password file <none>
I20260806 18:30:18.484198 19314 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:18.484265 19314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:18.484474 19314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:18.486536 19314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "b22221be5e5d4943a659326dce4aba61"
format_stamp: "Formatted at 2026-08-06 18:30:18 on dist-test-slave-5bn2"
server_key: "cc3b335b9d35bcf0e42504190e5c4cba"
server_key_iv: "42ff8e746786c7c2ade14f97ab0b2cc9"
server_key_version: "encryptionkey@0"
I20260806 18:30:18.487090 19314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "b22221be5e5d4943a659326dce4aba61"
format_stamp: "Formatted at 2026-08-06 18:30:18 on dist-test-slave-5bn2"
server_key: "cc3b335b9d35bcf0e42504190e5c4cba"
server_key_iv: "42ff8e746786c7c2ade14f97ab0b2cc9"
server_key_version: "encryptionkey@0"
I20260806 18:30:18.490753 19314 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:30:18.493108 19333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:18.494295 19314 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:30:18.494429 19314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "b22221be5e5d4943a659326dce4aba61"
format_stamp: "Formatted at 2026-08-06 18:30:18 on dist-test-slave-5bn2"
server_key: "cc3b335b9d35bcf0e42504190e5c4cba"
server_key_iv: "42ff8e746786c7c2ade14f97ab0b2cc9"
server_key_version: "encryptionkey@0"
I20260806 18:30:18.494526 19314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:18.515970 19314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:18.519081 19314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:18.519270 19314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:18.519897 19314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:18.520748 19314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:18.520798 19314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:18.520853 19314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:18.520892 19314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:18.529309 19314 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:46055
I20260806 18:30:18.529335 19495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:46055 every 8 connection(s)
I20260806 18:30:18.530232 19314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:30:18.535181 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 19314
I20260806 18:30:18.535324 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:30:18.535703 7439 external_mini_cluster.cc:1428] Setting key e6111971b71f96dace0f2e3324766690
Aug 06 18:30:18 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041018, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:30:18.544353 19285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:18.532085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:53699 (local address 127.7.67.254:34483)
0806 18:30:18.532253 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:18.532258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:18.533266 (+ 1008us) server_negotiation.cc:407] Connection header received
0806 18:30:18.534158 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:18.534161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:18.534211 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:18.534273 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:18.535697 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:18.536335 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:18.537723 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:18.537952 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:18.538039 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:18.540944 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:18.540947 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:18.540949 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:18.540981 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:18.542280 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:18.542780 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:18.542783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:18.542784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:18.542897 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:18.543240 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:18.543244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:18.543245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:18.543445 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:18.543535 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:18.544046 (+ 511us) server_negotiation.cc:299] Negotiation successful
0806 18:30:18.544181 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:30:18.545202 19498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:18.532430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.193:53699)
0806 18:30:18.533119 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:18.533164 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:18.533964 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:18.534436 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:18.534442 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:18.534808 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:18.535537 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:18.535560 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:18.536463 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:18.536466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:18.537594 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:18.537607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:18.538113 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:18.538117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:18.538315 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:18.538941 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:18.538962 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:18.540773 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:18.542394 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:18.542400 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:18.542416 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:18.542683 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:18.542987 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:18.542991 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:18.542993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:18.543148 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:18.543547 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:18.543555 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:18.543810 (+ 255us) client_negotiation.cc:769] Sending connection context
0806 18:30:18.543981 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 18:30:18.544318 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":107,"threads_started":1}
WARNING: no policy specified for kudu/127.7.67.194@KRBTEST.COM; defaulting to no policy
I20260806 18:30:18.546422 19496 heartbeater.cc:347] Connected to a master server at 127.7.67.254:34483
I20260806 18:30:18.546654 19496 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:18.547194 19496 heartbeater.cc:511] Master 127.7.67.254:34483 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.194@KRBTEST.COM" created.
I20260806 18:30:18.548808 19227 ts_manager.cc:194] Registered new tserver with Master: b22221be5e5d4943a659326dce4aba61 (127.7.67.193:46055)
I20260806 18:30:18.550263 19227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:53699
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
I20260806 18:30:18.565644 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:34483
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:41565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:18.680665 19501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:18.680925 19501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:18.680986 19501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:18.684973 19501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:18.685034 19501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:18.685113 19501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:30:18.690305 19501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:41565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:34483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:18.691391 19501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:18.692399 19501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:18.701977 19509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:18.701987 19508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:18.701977 19511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:18.702252 19501 server_base.cc:1034] running on GCE node
I20260806 18:30:18.702692 19501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:18.703333 19501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:18.704521 19501 hybrid_clock.cc:648] HybridClock initialized: now 1786041018704501 us; error 33 us; skew 500 ppm
Aug 06 18:30:18 dist-test-slave-5bn2 krb5kdc[19163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041018, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:18.707213 19501 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:30:18.708048 19501 webserver.cc:460] Webserver started at http://127.7.67.194:36567/ using document root <none> and password file <none>
I20260806 18:30:18.708575 19501 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:18.708632 19501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:18.708796 19501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:18.710776 19501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "641aabcc6fec4712ace8a833fc508e86"
format_stamp: "Formatted at 2026-08-06 18:30:18 on dist-test-slave-5bn2"
server_key: "18bb23ddb25ccba7f79f385fc5ffb086"
server_key_iv: "8b6b32f7543929cbc464460281a164a7"
server_key_version: "encryptionkey@0"
I20260806 18:30:18.711326 19501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "641aabcc6fec4712ace8a833fc508e86"
format_stamp: "Formatted at 2026-08-06 18:30:18 on dist-test-slave-5bn2"
server_key: "18bb23ddb25ccba7f79f385fc5ffb086"
server_key_iv: "8b6b32f7543929cbc464460281a164a7"
server_key_version: "encryptionkey@0"
I20260806 18:30:18.714813 19501 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:30:18.717200 19519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:18.718425 19501 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:30:18.718564 19501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "641aabcc6fec4712ace8a833fc508e86"
format_stamp: "Formatted at 2026-08-06 18:30:18 on dist-test-slave-5bn2"
server_key: "18bb23ddb25ccba7f79f385fc5ffb086"
server_key_iv: "8b6b32f7543929cbc464460281a164a7"
server_key_version: "encryptionkey@0"
I20260806 18:30:18.718657 19501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:18.728204 19501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:18.731253 19501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:18.731423 19501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:18.731986 19501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:18.732792 19501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:18.732841 19501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:18.732882 19501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:18.732911 19501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:18.740989 19501 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:39873
I20260806 18:30:18.741056 19672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:39873 every 8 connection(s)
I20260806 18:30:18.741861 19501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:30:18.742686 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 19501
I20260806 18:30:18.742800 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:30:18.743103 7439 external_mini_cluster.cc:1428] Setting key 329109f79876e18dddb51275efd59aac
Aug 06 18:30:18 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041018, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.195@KRBTEST.COM" created.
I20260806 18:30:18.756686 19285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:18.743902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:46381 (local address 127.7.67.254:34483)
0806 18:30:18.744027 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:18.744031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:18.745018 (+ 987us) server_negotiation.cc:407] Connection header received
0806 18:30:18.746108 (+ 1090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:18.746111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:18.746158 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:18.746224 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:18.747621 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:18.748372 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:18.749985 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:18.750265 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:18.750344 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:18.753186 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:18.753189 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:18.753191 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:18.753216 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:18.754602 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:18.755159 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:18.755163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:18.755165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:18.755256 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:18.755536 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:18.755538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:18.755539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:18.755715 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:18.755785 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:18.756363 (+ 578us) server_negotiation.cc:299] Negotiation successful
0806 18:30:18.756514 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:30:18.757516 19675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:18.744116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.194:46381)
0806 18:30:18.744855 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:18.744912 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:18.745871 (+ 959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:18.746421 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:18.746431 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:18.746945 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:18.747465 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:18.747479 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:18.748517 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:18.748520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:18.749869 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:18.749881 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:18.750421 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:18.750425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:18.750628 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:18.751234 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:18.751257 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:18.752993 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:18.754710 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:18.754715 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:18.754728 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:18.755071 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:18.755326 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:18.755328 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:18.755329 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:18.755463 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:18.755787 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:18.755791 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:18.756074 (+ 283us) client_negotiation.cc:769] Sending connection context
0806 18:30:18.756265 (+ 191us) client_negotiation.cc:240] Negotiation successful
0806 18:30:18.756662 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":119,"threads_started":1}
I20260806 18:30:18.758662 19673 heartbeater.cc:347] Connected to a master server at 127.7.67.254:34483
I20260806 18:30:18.758963 19673 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:18.759472 19673 heartbeater.cc:511] Master 127.7.67.254:34483 requested a full tablet report, sending...
I20260806 18:30:18.760603 19227 ts_manager.cc:194] Registered new tserver with Master: 641aabcc6fec4712ace8a833fc508e86 (127.7.67.194:39873)
I20260806 18:30:18.761385 19227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:46381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
I20260806 18:30:18.771571 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:34483
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:41565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:18.884930 19679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:18.885198 19679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:18.885253 19679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:18.889190 19679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:18.889249 19679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:18.889325 19679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:30:18.894495 19679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:41565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:34483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:18.895574 19679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:18.896602 19679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:18.906392 19688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:18.906375 19685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:18.906450 19686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:18.906781 19679 server_base.cc:1034] running on GCE node
I20260806 18:30:18.907270 19679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:18.907917 19679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:18.909104 19679 hybrid_clock.cc:648] HybridClock initialized: now 1786041018909084 us; error 28 us; skew 500 ppm
Aug 06 18:30:18 dist-test-slave-5bn2 krb5kdc[19163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041018, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:18.911979 19679 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:30:18.912914 19679 webserver.cc:460] Webserver started at http://127.7.67.195:42997/ using document root <none> and password file <none>
I20260806 18:30:18.913473 19679 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:18.913542 19679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:18.913744 19679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:18.915993 19679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "7f139e3b16924c679682bfa39a9c2bf0"
format_stamp: "Formatted at 2026-08-06 18:30:18 on dist-test-slave-5bn2"
server_key: "eebd89593bc2a20efd9bb22d42608ff8"
server_key_iv: "872a49cc3e3fd54ca6a4b6c0689a7fbd"
server_key_version: "encryptionkey@0"
I20260806 18:30:18.916576 19679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "7f139e3b16924c679682bfa39a9c2bf0"
format_stamp: "Formatted at 2026-08-06 18:30:18 on dist-test-slave-5bn2"
server_key: "eebd89593bc2a20efd9bb22d42608ff8"
server_key_iv: "872a49cc3e3fd54ca6a4b6c0689a7fbd"
server_key_version: "encryptionkey@0"
I20260806 18:30:18.920873 19679 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:30:18.923516 19699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:18.924716 19679 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:30:18.924820 19679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "7f139e3b16924c679682bfa39a9c2bf0"
format_stamp: "Formatted at 2026-08-06 18:30:18 on dist-test-slave-5bn2"
server_key: "eebd89593bc2a20efd9bb22d42608ff8"
server_key_iv: "872a49cc3e3fd54ca6a4b6c0689a7fbd"
server_key_version: "encryptionkey@0"
I20260806 18:30:18.924911 19679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:18.967952 19679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:18.971122 19679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:18.971287 19679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:18.971858 19679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:18.972692 19679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:18.972743 19679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:18.972784 19679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:18.972797 19679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:18.981333 19679 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:42231
I20260806 18:30:18.981372 19846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:42231 every 8 connection(s)
I20260806 18:30:18.982246 19679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:30:18.988617 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 19679
I20260806 18:30:18.988768 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:30:18.989090 7439 external_mini_cluster.cc:1428] Setting key c497a37311e88824d7b19807684aa5d2
Aug 06 18:30:18 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041018, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:30:18.995553 19285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:18.984056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:35099 (local address 127.7.67.254:34483)
0806 18:30:18.984192 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:18.984196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:18.985293 (+ 1097us) server_negotiation.cc:407] Connection header received
0806 18:30:18.986180 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:18.986184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:18.986243 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:18.986320 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:18.987557 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:18.988093 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:18.989612 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:18.989869 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:18.989964 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:18.992510 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:18.992514 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:18.992516 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:18.992541 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:18.993733 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:18.994173 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:18.994175 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:18.994176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:18.994244 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:18.994507 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:18.994509 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:18.994510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:18.994691 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:18.994762 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:18.995311 (+ 549us) server_negotiation.cc:299] Negotiation successful
0806 18:30:18.995417 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:30:18.996349 19851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:18.984454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.195:35099)
0806 18:30:18.985142 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:18.985185 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:18.985967 (+ 782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:18.986499 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:18.986507 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:18.986901 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:18.987406 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:18.987420 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:18.988216 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:18.988219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:18.989494 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:18.989505 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:18.990035 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:18.990040 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:18.990214 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:18.990798 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:18.990816 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:18.992334 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:18.993834 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:18.993839 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:18.993852 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:18.994099 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:18.994304 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:18.994306 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:18.994307 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:18.994438 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:18.994793 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:18.994799 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:18.995102 (+ 303us) client_negotiation.cc:769] Sending connection context
0806 18:30:18.995260 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 18:30:18.995528 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":126,"threads_started":1}
I20260806 18:30:18.997403 19848 heartbeater.cc:347] Connected to a master server at 127.7.67.254:34483
I20260806 18:30:18.997632 19848 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:18.998158 19848 heartbeater.cc:511] Master 127.7.67.254:34483 requested a full tablet report, sending...
I20260806 18:30:18.999248 19227 ts_manager.cc:194] Registered new tserver with Master: 7f139e3b16924c679682bfa39a9c2bf0 (127.7.67.195:42231)
I20260806 18:30:18.999788 19227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:35099
I20260806 18:30:19.002842 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.193.keytab.
Entry for principal oryx/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.193.keytab.
WARNING: no policy specified for oryx/127.7.67.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.194.keytab.
Entry for principal oryx/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.194.keytab.
WARNING: no policy specified for oryx/127.7.67.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.195.keytab.
Entry for principal oryx/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.195.keytab.
I20260806 18:30:19.077405 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 19314
I20260806 18:30:19.088775 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 19501
I20260806 18:30:19.101992 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 19679
I20260806 18:30:19.109565 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 19181
I20260806 18:30:19.127544 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34483
--webserver_interface=127.7.67.254
--webserver_port=40275
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:41565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:34483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:19.239044 19859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:19.239315 19859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:19.239356 19859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:19.243301 19859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:30:19.243360 19859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:19.243377 19859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:19.243396 19859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:30:19.243423 19859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:30:19.248678 19859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:41565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:34483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=40275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:19.249756 19859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:19.250797 19859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:19.259123 19870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:19.259183 19867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:19.259269 19859 server_base.cc:1034] running on GCE node
W20260806 18:30:19.259220 19868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:19.259913 19859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:19.260998 19859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:19.262163 19859 hybrid_clock.cc:648] HybridClock initialized: now 1786041019262143 us; error 29 us; skew 500 ppm
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041019, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:19.264990 19859 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:30:19.265877 19859 webserver.cc:460] Webserver started at http://127.7.67.254:40275/ using document root <none> and password file <none>
I20260806 18:30:19.266410 19859 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:19.266489 19859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:19.270295 19859 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:30:19.272445 19877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:19.273689 19859 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:30:19.273806 19859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "ed34b4cd1a68472f80d938d1ec5298ff"
format_stamp: "Formatted at 2026-08-06 18:30:18 on dist-test-slave-5bn2"
server_key: "3d11e8d1dea30639bb06cc8a29e00d74"
server_key_iv: "9bc64420d1d294bba1309bf1d5685b59"
server_key_version: "encryptionkey@0"
I20260806 18:30:19.274338 19859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:19.290663 19859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:19.293483 19859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:19.293669 19859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:19.300910 19859 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:34483
I20260806 18:30:19.300938 19953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:34483 every 8 connection(s)
I20260806 18:30:19.302011 19859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:30:19.304003 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 19859
I20260806 18:30:19.304517 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:46055
--local_ip_for_outbound_sockets=127.7.67.193
--tserver_master_addrs=127.7.67.254:34483
--webserver_port=43157
--webserver_interface=127.7.67.193
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:41565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.67.193
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:30:19.307140 19955 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:30:19.308916 19955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff: Bootstrap starting.
I20260806 18:30:19.326200 19955 log.cc:824] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:19.332924 19955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:30:19.333290 19955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff: Bootstrap complete.
I20260806 18:30:19.337492 19955 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed34b4cd1a68472f80d938d1ec5298ff" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34483 } }
I20260806 18:30:19.337891 19955 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed34b4cd1a68472f80d938d1ec5298ff, State: Initialized, Role: FOLLOWER
I20260806 18:30:19.338392 19955 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed34b4cd1a68472f80d938d1ec5298ff" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34483 } }
I20260806 18:30:19.338516 19955 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:30:19.338579 19955 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:30:19.338675 19955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:30:19.340703 19955 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed34b4cd1a68472f80d938d1ec5298ff" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34483 } }
I20260806 18:30:19.341030 19955 leader_election.cc:302] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed34b4cd1a68472f80d938d1ec5298ff; no voters:
I20260806 18:30:19.341337 19955 leader_election.cc:288] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:30:19.341491 19960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:30:19.341744 19960 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [term 2 LEADER]: Becoming Leader. State: Replica: ed34b4cd1a68472f80d938d1ec5298ff, State: Running, Role: LEADER
I20260806 18:30:19.342132 19960 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed34b4cd1a68472f80d938d1ec5298ff" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34483 } }
I20260806 18:30:19.342530 19955 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:30:19.343735 19962 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ed34b4cd1a68472f80d938d1ec5298ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed34b4cd1a68472f80d938d1ec5298ff" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34483 } } }
I20260806 18:30:19.343881 19962 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:19.344586 19976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:30:19.344799 19964 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed34b4cd1a68472f80d938d1ec5298ff. Latest consensus state: current_term: 2 leader_uuid: "ed34b4cd1a68472f80d938d1ec5298ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed34b4cd1a68472f80d938d1ec5298ff" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34483 } } }
I20260806 18:30:19.345000 19964 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:19.347826 19976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:30:19.348474 19976 catalog_manager.cc:1259] Loaded cluster ID: 4e2cae676bba4865beb03485eb810f74
I20260806 18:30:19.348549 19976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:30:19.350991 19976 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:30:19.351670 19976 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ed34b4cd1a68472f80d938d1ec5298ff: Loaded TSK: 0
I20260806 18:30:19.352901 19976 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:30:19.420532 19958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:19.420810 19958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:19.420871 19958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:19.424907 19958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:19.424978 19958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:19.425056 19958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:30:19.430385 19958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:41565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.193.keytab
--principal=oryx/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:46055
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=43157
--tserver_master_addrs=127.7.67.254:34483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:19.431486 19958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:19.432512 19958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:19.441983 19996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:19.441972 19993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:19.441967 19992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:19.442255 19958 server_base.cc:1034] running on GCE node
I20260806 18:30:19.442799 19958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:19.443652 19958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:19.444919 19958 hybrid_clock.cc:648] HybridClock initialized: now 1786041019444896 us; error 37 us; skew 500 ppm
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041019, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:19.448586 19958 init.cc:377] Logged in from keytab as oryx/127.7.67.193@KRBTEST.COM (short username oryx)
I20260806 18:30:19.449808 19958 webserver.cc:460] Webserver started at http://127.7.67.193:43157/ using document root <none> and password file <none>
I20260806 18:30:19.450582 19958 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:19.450703 19958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:19.456185 19958 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260806 18:30:19.458954 20005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:19.460479 19958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:30:19.460629 19958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "b22221be5e5d4943a659326dce4aba61"
format_stamp: "Formatted at 2026-08-06 18:30:18 on dist-test-slave-5bn2"
server_key: "cc3b335b9d35bcf0e42504190e5c4cba"
server_key_iv: "42ff8e746786c7c2ade14f97ab0b2cc9"
server_key_version: "encryptionkey@0"
I20260806 18:30:19.461356 19958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:19.491842 19958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:19.494982 19958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:19.495168 19958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:19.495779 19958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:19.496635 19958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:19.496685 19958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:19.496740 19958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:19.496768 19958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:19.505331 19958 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:46055
I20260806 18:30:19.505359 20163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:46055 every 8 connection(s)
I20260806 18:30:19.506341 19958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:30:19.512503 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 19958
I20260806 18:30:19.513010 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:39873
--local_ip_for_outbound_sockets=127.7.67.194
--tserver_master_addrs=127.7.67.254:34483
--webserver_port=36567
--webserver_interface=127.7.67.194
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:41565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.67.194
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:30:19.519804 20167 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:19.508366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:57491 (local address 127.7.67.254:34483)
0806 18:30:19.508808 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:19.508824 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:19.509299 (+ 475us) server_negotiation.cc:407] Connection header received
0806 18:30:19.510361 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:19.510374 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:19.510669 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:19.511021 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:19.512043 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.513455 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.514642 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.515190 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.515313 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:19.519083 (+ 3770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.193:57491: BlockingRecv error: recv got EOF from 127.7.67.193:57491 (error 108)
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":104,"threads_started":1}
I20260806 18:30:19.519896 20168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:19.508446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.193:57491)
0806 18:30:19.509143 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:19.509194 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:19.509974 (+ 780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:19.511066 (+ 1092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:19.511073 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:19.511446 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:19.511909 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:19.511923 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.513603 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.513606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:19.514483 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:19.514495 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.515354 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.515359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:19.515571 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:19.516205 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:19.516224 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:19.518834 (+ 2610us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":120,"threads_started":1}
W20260806 18:30:19.520717 20164 heartbeater.cc:644] Failed to heartbeat to 127.7.67.254:34483 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.67.254:34483: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:19.526039 20168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:19.521206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.193:46955)
0806 18:30:19.521333 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:19.521347 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:19.521431 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:19.521660 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:19.521663 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:19.521819 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:19.521913 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:19.521919 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.522627 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.522630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:19.523209 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:19.523218 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.523629 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.523632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:19.523726 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:19.524177 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:19.524189 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:19.525897 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 18:30:19.526350 20167 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:19.521277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:46955 (local address 127.7.67.254:34483)
0806 18:30:19.521382 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:19.521386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:19.521399 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:30:19.521518 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:19.521521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:19.521566 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:19.521631 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:19.522028 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.522527 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.523316 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.523531 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.523618 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:19.526038 (+ 2420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.193:46955: BlockingRecv error: recv got EOF from 127.7.67.193:46955 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:19.531231 20168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:19.526521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.193:45025)
0806 18:30:19.526628 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:19.526643 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:19.526714 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:19.526943 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:19.526945 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:19.527066 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:19.527144 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:19.527149 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.528116 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.528118 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:19.528620 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:19.528627 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.529129 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.529132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:19.529183 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:19.529593 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:19.529605 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:19.531139 (+ 1534us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 18:30:19.531395 20167 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:19.526627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:45025 (local address 127.7.67.254:34483)
0806 18:30:19.526742 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:19.526746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:19.526760 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:30:19.526795 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:19.526798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:19.526839 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:19.526925 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:19.527233 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.528029 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.528709 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.529048 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.529138 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:19.531286 (+ 2148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.193:45025: BlockingRecv error: recv got EOF from 127.7.67.193:45025 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 18:30:19.531662 20164 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:30:19.628927 20169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:19.629199 20169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:19.629241 20169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:19.633244 20169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:19.633307 20169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:19.633365 20169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:30:19.638795 20169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:41565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.194.keytab
--principal=oryx/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:39873
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=36567
--tserver_master_addrs=127.7.67.254:34483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:19.639918 20169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:19.640921 20169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:19.650830 20177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:19.650825 20176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:19.650820 20179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:19.651371 20169 server_base.cc:1034] running on GCE node
I20260806 18:30:19.651791 20169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:19.652485 20169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:19.653663 20169 hybrid_clock.cc:648] HybridClock initialized: now 1786041019653634 us; error 38 us; skew 500 ppm
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041019, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:19.656584 20169 init.cc:377] Logged in from keytab as oryx/127.7.67.194@KRBTEST.COM (short username oryx)
I20260806 18:30:19.657476 20169 webserver.cc:460] Webserver started at http://127.7.67.194:36567/ using document root <none> and password file <none>
I20260806 18:30:19.658047 20169 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:19.658135 20169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:19.662133 20169 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:30:19.664335 20189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:19.665704 20169 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:30:19.665824 20169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "641aabcc6fec4712ace8a833fc508e86"
format_stamp: "Formatted at 2026-08-06 18:30:18 on dist-test-slave-5bn2"
server_key: "18bb23ddb25ccba7f79f385fc5ffb086"
server_key_iv: "8b6b32f7543929cbc464460281a164a7"
server_key_version: "encryptionkey@0"
I20260806 18:30:19.666437 20169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:19.683732 20169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:19.686641 20169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:19.686823 20169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:19.687462 20169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:19.688308 20169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:19.688357 20169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:19.688414 20169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:19.688438 20169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:19.697073 20169 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:39873
I20260806 18:30:19.697160 20349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:39873 every 8 connection(s)
I20260806 18:30:19.698108 20169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:30:19.700567 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 20169
I20260806 18:30:19.701001 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:42231
--local_ip_for_outbound_sockets=127.7.67.195
--tserver_master_addrs=127.7.67.254:34483
--webserver_port=42997
--webserver_interface=127.7.67.195
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:41565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.67.195
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:30:19.709506 20167 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:19.699940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:37993 (local address 127.7.67.254:34483)
0806 18:30:19.700066 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:19.700070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:19.701179 (+ 1109us) server_negotiation.cc:407] Connection header received
0806 18:30:19.702148 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:19.702152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:19.702216 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:19.702308 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:19.703877 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.704429 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.705579 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.705804 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.705894 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:19.709358 (+ 3464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.194:37993: BlockingRecv error: recv got EOF from 127.7.67.194:37993 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:30:19.710008 20353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:19.700295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.194:37993)
0806 18:30:19.701015 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:19.701070 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:19.701927 (+ 857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:19.702483 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:19.702490 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:19.703002 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:19.703693 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:19.703714 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.704547 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.704549 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:19.705440 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:19.705460 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.705955 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.705959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:19.706131 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:19.706684 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:19.706701 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:19.709167 (+ 2466us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":118,"threads_started":1}
W20260806 18:30:19.710603 20350 heartbeater.cc:644] Failed to heartbeat to 127.7.67.254:34483 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.67.254:34483: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:19.716348 20353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:19.711075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.194:52225)
0806 18:30:19.711224 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:19.711239 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:19.711321 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:19.711566 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:19.711569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:19.711789 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:19.711935 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:19.711944 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.712627 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.712631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:19.713547 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:19.713558 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.713922 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.713925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:19.714015 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:19.714454 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:19.714468 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:19.716223 (+ 1755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":65}
W20260806 18:30:19.716471 20167 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:19.711199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:52225 (local address 127.7.67.254:34483)
0806 18:30:19.711298 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:19.711301 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:19.711313 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:30:19.711411 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:19.711413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:19.711462 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:19.711523 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:19.712036 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.712528 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.713646 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.713838 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.713902 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:19.716365 (+ 2463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.194:52225: BlockingRecv error: recv got EOF from 127.7.67.194:52225 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:19.721800 20353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:19.716996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.194:60105)
0806 18:30:19.717150 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:19.717173 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:19.717232 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:19.717424 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:19.717426 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:19.717555 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:19.717626 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:19.717630 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.718579 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.718582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:19.719116 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:19.719123 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.719635 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.719637 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:19.719689 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:19.720092 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:19.720104 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:19.721676 (+ 1572us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95}
W20260806 18:30:19.722047 20350 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:30:19.722167 20167 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:19.717099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:60105 (local address 127.7.67.254:34483)
0806 18:30:19.717222 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:19.717226 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:19.717240 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:30:19.717296 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:19.717299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:19.717339 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:19.717415 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:19.717710 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.718496 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.719217 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.719540 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.719635 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:19.721792 (+ 2157us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.194:60105: BlockingRecv error: recv got EOF from 127.7.67.194:60105 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 18:30:19.818723 20357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:19.819016 20357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:19.819072 20357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:19.823030 20357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:19.823084 20357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:19.823158 20357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:30:19.828439 20357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:41565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/krb5kdc/oryx_127.7.67.195.keytab
--principal=oryx/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:42231
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=42997
--tserver_master_addrs=127.7.67.254:34483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:19.829442 20357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:19.830456 20357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:19.839962 20364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:19.839916 20363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:19.840085 20366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:19.840178 20357 server_base.cc:1034] running on GCE node
I20260806 18:30:19.840613 20357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:19.841233 20357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:19.842419 20357 hybrid_clock.cc:648] HybridClock initialized: now 1786041019842414 us; error 43 us; skew 500 ppm
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041019, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:19.845325 20357 init.cc:377] Logged in from keytab as oryx/127.7.67.195@KRBTEST.COM (short username oryx)
I20260806 18:30:19.846186 20357 webserver.cc:460] Webserver started at http://127.7.67.195:42997/ using document root <none> and password file <none>
I20260806 18:30:19.846771 20357 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:19.846868 20357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:19.850697 20357 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 18:30:19.852762 20373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:19.854058 20357 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:30:19.854168 20357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "7f139e3b16924c679682bfa39a9c2bf0"
format_stamp: "Formatted at 2026-08-06 18:30:18 on dist-test-slave-5bn2"
server_key: "eebd89593bc2a20efd9bb22d42608ff8"
server_key_iv: "872a49cc3e3fd54ca6a4b6c0689a7fbd"
server_key_version: "encryptionkey@0"
I20260806 18:30:19.854698 20357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:19.868789 20357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:19.871876 20357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:19.872066 20357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:19.872617 20357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:19.873417 20357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:19.873466 20357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:19.873518 20357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:19.873545 20357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:19.882956 20357 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:42231
I20260806 18:30:19.883112 20524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:42231 every 8 connection(s)
I20260806 18:30:19.884259 20357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:30:19.889076 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 20357
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:30:19.895855 20167 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:19.887000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:54541 (local address 127.7.67.254:34483)
0806 18:30:19.887239 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:19.887246 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:19.887503 (+ 257us) server_negotiation.cc:407] Connection header received
0806 18:30:19.888816 (+ 1313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:19.888819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:19.888864 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:19.888941 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:19.890370 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.890958 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.892067 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.892286 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.892368 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:19.895731 (+ 3363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.195:54541: BlockingRecv error: recv got EOF from 127.7.67.195:54541 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 18:30:19.896658 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:19.886815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.195:54541)
0806 18:30:19.887360 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:19.887413 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:19.888621 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:19.889137 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:19.889146 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:19.889611 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:19.890193 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:19.890213 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.891072 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.891075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:19.891954 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:19.891963 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.892417 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.892422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:19.892587 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:19.893160 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:19.893181 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:19.895617 (+ 2436us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":146,"threads_started":1}
W20260806 18:30:19.897145 20525 heartbeater.cc:644] Failed to heartbeat to 127.7.67.254:34483 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.67.254:34483: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
I20260806 18:30:19.897980 20529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:19.890114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55714 (local address 127.7.67.254:34483)
0806 18:30:19.890312 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:19.890315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:19.890326 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:30:19.890361 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:19.890363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:19.890396 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:19.890493 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:19.891270 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.891779 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.892599 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.892826 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.892921 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:19.894259 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:19.894266 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:19.894284 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:19.894316 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:19.895902 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:19.896373 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:19.896378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:19.896382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:19.896504 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:19.896799 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:19.896802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:19.896804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:19.897235 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:19.897448 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:19.897642 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 18:30:19.897832 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":60,"threads_started":1}
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:19.902649 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:19.897591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.195:57743)
0806 18:30:19.897912 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:19.897934 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:19.898013 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:19.898329 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:19.898331 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:19.898480 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:19.898588 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:19.898594 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.899475 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.899478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:19.900148 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:19.900159 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.900673 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.900677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:19.900768 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:19.901115 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:19.901126 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:19.902539 (+ 1413us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260}
W20260806 18:30:19.902748 20167 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:19.897802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:57743 (local address 127.7.67.254:34483)
0806 18:30:19.897897 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:19.897901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:19.897992 (+ 91us) server_negotiation.cc:407] Connection header received
0806 18:30:19.898115 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:19.898118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:19.898159 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:19.898203 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:19.898705 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.899366 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.900264 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.900560 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.900655 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:19.902658 (+ 2003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.195:57743: BlockingRecv error: recv got EOF from 127.7.67.195:57743 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:19 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:19.907233 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:19.903137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.195:44905)
0806 18:30:19.903229 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:19.903241 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:19.903311 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:19.903544 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:19.903547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:19.903671 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:19.903736 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:19.903740 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.904390 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.904393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:19.904915 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:19.904923 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.905318 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.905322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:19.905392 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:19.905711 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:19.905721 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:19.907123 (+ 1402us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 18:30:19.907330 20167 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:19.903214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:44905 (local address 127.7.67.254:34483)
0806 18:30:19.903351 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:19.903356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:19.903367 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:30:19.903397 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:19.903400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:19.903456 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:19.903514 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:19.903833 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.904295 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.905015 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:19.905226 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:19.905306 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:19.907243 (+ 1937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.195:44905: BlockingRecv error: recv got EOF from 127.7.67.195:44905 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260806 18:30:19.907488 20525 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 18:30:20 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:20 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:20.538255 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:20.532329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.193:46065)
0806 18:30:20.532617 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:20.532632 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:20.532728 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:20.532962 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:20.532964 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:20.533161 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:20.533268 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:20.533274 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:20.534068 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:20.534072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:20.534676 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:20.534687 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:20.535143 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:20.535145 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:20.535210 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:20.535776 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:20.535788 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:20.538088 (+ 2300us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":79,"threads_started":1}
W20260806 18:30:20.538412 20531 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:20.532426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:46065 (local address 127.7.67.254:34483)
0806 18:30:20.532661 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:20.532664 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:20.532679 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:30:20.532807 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:20.532809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:20.532854 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:20.532935 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:20.533379 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:20.533930 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:20.534793 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:20.535055 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:20.535136 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:20.538252 (+ 3116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.193:46065: BlockingRecv error: recv got EOF from 127.7.67.193:46065 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":72,"threads_started":1}
Aug 06 18:30:20 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:20 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:20.728195 20532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:20.722737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.194:60533)
0806 18:30:20.723036 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:20.723051 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:20.723162 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:20.723393 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:20.723395 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:20.723601 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:20.723709 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:20.723715 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:20.724473 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:20.724477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:20.725190 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:20.725201 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:20.725733 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:20.725736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:20.725801 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:20.726263 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:20.726273 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:20.728069 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":110,"threads_started":1}
W20260806 18:30:20.728334 20531 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:20.722822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:60533 (local address 127.7.67.254:34483)
0806 18:30:20.722966 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:20.722969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:20.723119 (+ 150us) server_negotiation.cc:407] Connection header received
0806 18:30:20.723240 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:20.723242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:20.723286 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:20.723360 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:20.723804 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:20.724352 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:20.725298 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:20.725603 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:20.725697 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:20.728210 (+ 2513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.194:60533: BlockingRecv error: recv got EOF from 127.7.67.194:60533 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 18:30:20 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:20 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:20.913473 20533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:20.908119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.195:57725)
0806 18:30:20.908416 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:20.908432 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:20.908549 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:20.908795 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:20.908798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:20.908997 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:20.909102 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:20.909108 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:20.909852 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:20.909856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:20.910452 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:20.910460 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:20.910928 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:20.910932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:20.911009 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:20.911503 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:20.911515 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:20.913313 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":100,"threads_started":1}
W20260806 18:30:20.913579 20531 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:20.908260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:57725 (local address 127.7.67.254:34483)
0806 18:30:20.908402 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:20.908407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:20.908504 (+ 97us) server_negotiation.cc:407] Connection header received
0806 18:30:20.908638 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:20.908641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:20.908694 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:20.908760 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:20.909216 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:20.909749 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:20.910571 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:20.910799 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:20.910915 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:20.913465 (+ 2550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.195:57725: BlockingRecv error: recv got EOF from 127.7.67.195:57725 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 18:30:21 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:21 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:21.544634 20534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:21.539156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.193:54285)
0806 18:30:21.539489 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:21.539504 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:21.539601 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:21.539823 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:21.539824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:21.540010 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:21.540111 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:21.540116 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:21.540913 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:21.540916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:21.541480 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:21.541489 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:21.541917 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:21.541919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:21.541976 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:21.542487 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:21.542513 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:21.544474 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":98,"threads_started":1}
W20260806 18:30:21.544780 20535 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:21.539275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:54285 (local address 127.7.67.254:34483)
0806 18:30:21.539555 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:21.539559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:21.539576 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:30:21.539672 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:21.539674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:21.539721 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:21.539802 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:21.540204 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:21.540776 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:21.541590 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:21.541829 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:21.541911 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:21.544636 (+ 2725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.193:54285: BlockingRecv error: recv got EOF from 127.7.67.193:54285 (error 108)
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":86,"threads_started":1}
Aug 06 18:30:21 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:21 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:21.734205 20538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:21.728954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.194:36343)
0806 18:30:21.729228 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:21.729242 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:21.729350 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:21.729623 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:21.729626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:21.729817 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:21.729939 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:21.729946 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:21.730658 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:21.730661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:21.731302 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:21.731313 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:21.731683 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:21.731685 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:21.731745 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:21.732257 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:21.732269 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:21.734048 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":78,"threads_started":1}
W20260806 18:30:21.734344 20535 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:21.729037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:36343 (local address 127.7.67.254:34483)
0806 18:30:21.729168 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:21.729172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:21.729304 (+ 132us) server_negotiation.cc:407] Connection header received
0806 18:30:21.729416 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:21.729418 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:21.729465 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:21.729540 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:21.730038 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:21.730570 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:21.731395 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:21.731602 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:21.731668 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:21.734210 (+ 2542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.194:36343: BlockingRecv error: recv got EOF from 127.7.67.194:36343 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 18:30:21 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:21 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:21.922494 20539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:21.914314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.195:47507)
0806 18:30:21.914591 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:21.914604 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:21.914704 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:21.914943 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:21.914945 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:21.915129 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:21.915231 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:21.915236 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:21.915916 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:21.915918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:21.916493 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:21.916500 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:21.916861 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:21.916863 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:21.916920 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:21.920438 (+ 3518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:21.920453 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:21.922285 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":92,"threads_started":1}
W20260806 18:30:21.922560 20535 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:21.914439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:47507 (local address 127.7.67.254:34483)
0806 18:30:21.914568 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:21.914572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:21.914652 (+ 80us) server_negotiation.cc:407] Connection header received
0806 18:30:21.914769 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:21.914771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:21.914813 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:21.914926 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:21.915318 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:21.915834 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:21.916574 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:21.916778 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:21.916847 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:21.922450 (+ 5603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.195:47507: BlockingRecv error: recv got EOF from 127.7.67.195:47507 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 18:30:22 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:22 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:22.550484 20540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:22.545404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.193:58719)
0806 18:30:22.545686 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:22.545700 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:22.545800 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:22.546004 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:22.546006 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:22.546188 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:22.546292 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:22.546297 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:22.547015 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:22.547018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:22.547563 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:22.547570 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:22.547933 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:22.547935 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:22.547994 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:22.548442 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:22.548453 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:22.550352 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":90,"threads_started":1}
W20260806 18:30:22.550618 20541 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:22.545438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:58719 (local address 127.7.67.254:34483)
0806 18:30:22.545710 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:22.545714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:22.545763 (+ 49us) server_negotiation.cc:407] Connection header received
0806 18:30:22.545870 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:22.545873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:22.545916 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:22.545989 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:22.546374 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:22.546940 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:22.547648 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:22.547857 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:22.547919 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:22.550494 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.193:58719: BlockingRecv error: recv got EOF from 127.7.67.193:58719 (error 108)
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":106,"threads_started":1}
Aug 06 18:30:22 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:22 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:22.740265 20543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:22.734968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.194:38543)
0806 18:30:22.735270 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:22.735284 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:22.735385 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:22.735619 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:22.735620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:22.735803 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:22.735906 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:22.735912 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:22.736628 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:22.736630 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:22.737224 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:22.737233 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:22.737714 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:22.737717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:22.737773 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:22.738255 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:22.738267 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:22.740104 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":107,"threads_started":1}
W20260806 18:30:22.740404 20541 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:22.735079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:38543 (local address 127.7.67.254:34483)
0806 18:30:22.735230 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:22.735234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:22.735343 (+ 109us) server_negotiation.cc:407] Connection header received
0806 18:30:22.735472 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:22.735474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:22.735520 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:22.735588 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:22.735987 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:22.736523 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:22.737384 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:22.737622 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:22.737712 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:22.740259 (+ 2547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.194:38543: BlockingRecv error: recv got EOF from 127.7.67.194:38543 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 18:30:22 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:22 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:22.929019 20544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:22.923370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.195:39381)
0806 18:30:22.923644 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:22.923657 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:22.923767 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:22.924033 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:22.924035 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:22.924218 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:22.924319 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:22.924323 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:22.925041 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:22.925043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:22.925619 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:22.925627 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:22.926139 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:22.926143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:22.926215 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:22.926756 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:22.926768 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:22.928895 (+ 2127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":74,"threads_started":1}
W20260806 18:30:22.929176 20541 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:22.923499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:39381 (local address 127.7.67.254:34483)
0806 18:30:22.923660 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:22.923664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:22.923829 (+ 165us) server_negotiation.cc:407] Connection header received
0806 18:30:22.923879 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:22.923882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:22.923936 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:22.924005 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:22.924423 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:22.924951 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:22.925751 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:22.925998 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:22.926096 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:22.929039 (+ 2943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.195:39381: BlockingRecv error: recv got EOF from 127.7.67.195:39381 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 06 18:30:23 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:23 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:23.556962 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:23.551342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.193:39025)
0806 18:30:23.551652 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:23.551669 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:23.551806 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:23.552084 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:23.552087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:23.552297 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:23.552399 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:23.552405 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:23.553160 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:23.553164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:23.553721 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:23.553729 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:23.554170 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:23.554173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:23.554256 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:23.554720 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:23.554731 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:23.556787 (+ 2056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":92,"threads_started":1}
W20260806 18:30:23.557067 20546 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:23.551423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:39025 (local address 127.7.67.254:34483)
0806 18:30:23.551686 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:23.551691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:23.551742 (+ 51us) server_negotiation.cc:407] Connection header received
0806 18:30:23.551913 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:23.551916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:23.551967 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:23.552044 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:23.552518 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:23.553055 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:23.553836 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:23.554065 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:23.554151 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:23.556948 (+ 2797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.193:39025: BlockingRecv error: recv got EOF from 127.7.67.193:39025 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":80,"threads_started":1}
Aug 06 18:30:23 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:23 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:23.746455 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:23.741184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.194:52479)
0806 18:30:23.741450 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:23.741464 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:23.741569 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:23.741815 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:23.741818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:23.742011 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:23.742135 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:23.742142 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:23.742915 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:23.742919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:23.743606 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:23.743617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:23.744020 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:23.744022 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:23.744084 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:23.744599 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:23.744611 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:23.746324 (+ 1713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":77,"threads_started":1}
W20260806 18:30:23.746922 20546 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:23.741268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:52479 (local address 127.7.67.254:34483)
0806 18:30:23.741402 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:23.741406 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:23.741527 (+ 121us) server_negotiation.cc:407] Connection header received
0806 18:30:23.741634 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:23.741636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:23.741680 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:23.741750 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:23.742233 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:23.742779 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:23.743705 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:23.743930 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:23.744010 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:23.746513 (+ 2503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.194:52479: BlockingRecv error: recv got EOF from 127.7.67.194:52479 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 18:30:23 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:23 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:23.934966 20550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:23.929757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.195:53623)
0806 18:30:23.930052 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:23.930066 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:23.930190 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:23.930412 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:23.930414 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:23.930612 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:23.930713 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:23.930718 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:23.931437 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:23.931440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:23.932005 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:23.932013 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:23.932480 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:23.932485 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:23.932574 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:23.933075 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:23.933087 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:23.934790 (+ 1703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":95,"threads_started":1}
W20260806 18:30:23.935073 20546 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:23.929846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:53623 (local address 127.7.67.254:34483)
0806 18:30:23.930005 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:23.930011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:23.930148 (+ 137us) server_negotiation.cc:407] Connection header received
0806 18:30:23.930265 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:23.930267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:23.930316 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:23.930383 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:23.930802 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:23.931344 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:23.932111 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:23.932349 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:23.932443 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:23.934965 (+ 2522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.195:53623: BlockingRecv error: recv got EOF from 127.7.67.195:53623 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 18:30:24 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:24 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:24.563337 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:24.557943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.193:43171)
0806 18:30:24.558331 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:24.558345 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:24.558440 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:24.558662 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:24.558665 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:24.558886 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:24.558994 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:24.559001 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:24.559770 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:24.559772 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:24.560338 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:24.560346 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:24.560750 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:24.560752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:24.560809 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:24.561286 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:24.561297 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:24.563174 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":122,"threads_started":1}
W20260806 18:30:24.563474 20551 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:24.557980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:43171 (local address 127.7.67.254:34483)
0806 18:30:24.558239 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:24.558244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:24.558410 (+ 166us) server_negotiation.cc:407] Connection header received
0806 18:30:24.558503 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:24.558505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:24.558549 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:24.558625 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:24.559087 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:24.559686 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:24.560428 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:24.560664 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:24.560736 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:24.563342 (+ 2606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.193:43171: BlockingRecv error: recv got EOF from 127.7.67.193:43171 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":85,"threads_started":1}
Aug 06 18:30:24 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:24 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:24.752431 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:24.747308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.194:44819)
0806 18:30:24.747580 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:24.747596 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:24.747690 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:24.747908 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:24.747909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:24.748102 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:24.748199 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:24.748204 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:24.748894 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:24.748896 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:24.749447 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:24.749454 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:24.749895 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:24.749897 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:24.749952 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:24.750441 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:24.750452 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:24.752263 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":77,"threads_started":1}
W20260806 18:30:24.752521 20551 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:24.747457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:44819 (local address 127.7.67.254:34483)
0806 18:30:24.747593 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:24.747597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:24.747647 (+ 50us) server_negotiation.cc:407] Connection header received
0806 18:30:24.747760 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:24.747763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:24.747805 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:24.747883 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:24.748282 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:24.748805 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:24.749564 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:24.749796 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:24.749893 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:24.752416 (+ 2523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.194:44819: BlockingRecv error: recv got EOF from 127.7.67.194:44819 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 18:30:24 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:24 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:24.941125 20555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:24.935850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.195:59933)
0806 18:30:24.936151 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:24.936164 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:24.936265 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:24.936530 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:24.936532 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:24.936723 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:24.936824 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:24.936829 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:24.937583 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:24.937585 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:24.938147 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:24.938152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:24.938587 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:24.938589 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:24.938646 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:24.939147 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:24.939159 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:24.940968 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":95,"threads_started":1}
W20260806 18:30:24.941213 20551 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:24.935954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:59933 (local address 127.7.67.254:34483)
0806 18:30:24.936079 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:24.936083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:24.936229 (+ 146us) server_negotiation.cc:407] Connection header received
0806 18:30:24.936338 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:24.936340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:24.936417 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:24.936491 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:24.936941 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:24.937499 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:24.938246 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:24.938499 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:24.938578 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:24.941112 (+ 2534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.195:59933: BlockingRecv error: recv got EOF from 127.7.67.195:59933 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 18:30:25 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:25 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:25.569655 20557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:25.564302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.193:43859)
0806 18:30:25.564635 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:25.564649 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:25.564758 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:25.564973 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:25.564975 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:25.565161 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:25.565264 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:25.565269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:25.565958 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:25.565960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:25.566582 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:25.566592 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:25.567077 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:25.567080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:25.567140 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:25.567672 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:25.567684 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:25.569511 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":108,"threads_started":1}
W20260806 18:30:25.569828 20560 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:25.564370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:43859 (local address 127.7.67.254:34483)
0806 18:30:25.564627 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:25.564630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:25.564703 (+ 73us) server_negotiation.cc:407] Connection header received
0806 18:30:25.564823 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:25.564825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:25.564871 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:25.564944 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:25.565347 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:25.565877 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:25.566689 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:25.566985 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:25.567071 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:25.569679 (+ 2608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.193:43859: BlockingRecv error: recv got EOF from 127.7.67.193:43859 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":67,"threads_started":1}
Aug 06 18:30:25 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:25 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:25.758455 20561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:25.753287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.194:56925)
0806 18:30:25.753573 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:25.753587 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:25.753698 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:25.753916 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:25.753917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:25.754104 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:25.754204 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:25.754209 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:25.754963 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:25.754967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:25.755530 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:25.755540 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:25.755918 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:25.755920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:25.755978 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:25.756451 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:25.756462 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:25.758293 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":94,"threads_started":1}
W20260806 18:30:25.758561 20560 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:25.753377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:56925 (local address 127.7.67.254:34483)
0806 18:30:25.753549 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:25.753554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:25.753655 (+ 101us) server_negotiation.cc:407] Connection header received
0806 18:30:25.753770 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:25.753772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:25.753819 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:25.753884 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:25.754283 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:25.754804 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:25.755625 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:25.755835 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:25.755902 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:25.758452 (+ 2550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.194:56925: BlockingRecv error: recv got EOF from 127.7.67.194:56925 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 18:30:25 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:25 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:25.948030 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:25.942131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.195:50233)
0806 18:30:25.942434 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:25.942448 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:25.942556 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:25.942783 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:25.942785 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:25.943019 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:25.943160 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:25.943167 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:25.943942 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:25.943946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:25.944533 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:25.944545 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:25.945129 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:25.945133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:25.945220 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:25.945902 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:25.945917 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:25.947859 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":89,"threads_started":1}
W20260806 18:30:25.948110 20560 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:25.942209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:50233 (local address 127.7.67.254:34483)
0806 18:30:25.942338 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:25.942342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:25.942511 (+ 169us) server_negotiation.cc:407] Connection header received
0806 18:30:25.942630 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:25.942633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:25.942681 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:25.942755 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:25.943258 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:25.943845 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:25.944654 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:25.945009 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:25.945116 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:25.948007 (+ 2891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.195:50233: BlockingRecv error: recv got EOF from 127.7.67.195:50233 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 18:30:26 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:26 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:26.576023 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:26.570451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.193:34845)
0806 18:30:26.570817 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:26.570831 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:26.570964 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:26.571202 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:26.571204 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:26.571392 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:26.571513 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:26.571518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:26.572213 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:26.572215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:26.572776 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:26.572783 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:26.573156 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:26.573158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:26.573219 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:26.573763 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:26.573776 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:26.575896 (+ 2120us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":93,"threads_started":1}
W20260806 18:30:26.576152 20565 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:26.570555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:34845 (local address 127.7.67.254:34483)
0806 18:30:26.570832 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:26.570836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:26.570907 (+ 71us) server_negotiation.cc:407] Connection header received
0806 18:30:26.571050 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:26.571052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:26.571099 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:26.571176 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:26.571600 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:26.572132 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:26.572855 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:26.573074 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:26.573139 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:26.576032 (+ 2893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.193:34845: BlockingRecv error: recv got EOF from 127.7.67.193:34845 (error 108)
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":92,"threads_started":1}
Aug 06 18:30:26 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:26 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:26.764148 20567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:26.759255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.194:54397)
0806 18:30:26.759595 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:26.759610 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:26.759712 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:26.759917 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:26.759919 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:26.760110 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:26.760210 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:26.760216 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:26.760893 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:26.760894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:26.761438 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:26.761446 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:26.761793 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:26.761796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:26.761853 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:26.762303 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:26.762314 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:26.764019 (+ 1705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":84,"threads_started":1}
W20260806 18:30:26.764286 20565 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:26.759340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:54397 (local address 127.7.67.254:34483)
0806 18:30:26.759480 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:26.759484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:26.759670 (+ 186us) server_negotiation.cc:407] Connection header received
0806 18:30:26.759776 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:26.759778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:26.759822 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:26.759888 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:26.760292 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:26.760811 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:26.761515 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:26.761718 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:26.761785 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:26.764164 (+ 2379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.194:54397: BlockingRecv error: recv got EOF from 127.7.67.194:54397 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 18:30:26 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:26 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:26.959635 20568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:26.948862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.195:46255)
0806 18:30:26.949187 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:26.949201 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:26.949305 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:26.949528 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:26.949530 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:26.949720 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:26.949817 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:26.949822 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:26.950507 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:26.950509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:26.951095 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:26.951103 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:26.951455 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:26.951457 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:26.951524 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:26.957703 (+ 6179us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:26.957717 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:26.959511 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":82,"threads_started":1}
W20260806 18:30:26.959774 20565 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:26.948932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:46255 (local address 127.7.67.254:34483)
0806 18:30:26.949079 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:26.949084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:26.949269 (+ 185us) server_negotiation.cc:407] Connection header received
0806 18:30:26.949391 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:26.949394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:26.949445 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:26.949501 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:26.949901 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:26.950426 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:26.951171 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:26.951378 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:26.951444 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:26.959651 (+ 8207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.195:46255: BlockingRecv error: recv got EOF from 127.7.67.195:46255 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 18:30:27 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:27 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:27.582257 20571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:27.576810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.193:41829)
0806 18:30:27.577130 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:27.577144 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:27.577237 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:27.577484 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:27.577486 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:27.577674 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:27.577779 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:27.577784 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:27.578561 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:27.578563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:27.579178 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:27.579187 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:27.579695 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:27.579698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:27.579763 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:27.580341 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:27.580352 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:27.582128 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":99,"threads_started":1}
W20260806 18:30:27.582441 20572 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:27.576963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:41829 (local address 127.7.67.254:34483)
0806 18:30:27.577184 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:27.577188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:27.577203 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:30:27.577319 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:27.577323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:27.577372 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:27.577457 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:27.577876 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:27.578467 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:27.579314 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:27.579590 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:27.579679 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:27.582289 (+ 2610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.193:41829: BlockingRecv error: recv got EOF from 127.7.67.193:41829 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":66,"threads_started":1}
Aug 06 18:30:27 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:27 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:27.770641 20574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:27.764913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.194:39379)
0806 18:30:27.765200 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:27.765213 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:27.765317 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:27.765746 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:27.765749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:27.765991 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:27.766118 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:27.766125 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:27.767129 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:27.767133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:27.767700 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:27.767709 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:27.768140 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:27.768142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:27.768204 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:27.768706 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:27.768717 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:27.770476 (+ 1759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":102,"threads_started":1}
W20260806 18:30:27.770728 20572 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:27.765025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:39379 (local address 127.7.67.254:34483)
0806 18:30:27.765196 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:27.765200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:27.765275 (+ 75us) server_negotiation.cc:407] Connection header received
0806 18:30:27.765388 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:27.765390 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:27.765443 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:27.765691 (+ 248us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:27.766281 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:27.767016 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:27.767808 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:27.768044 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:27.768124 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:27.770621 (+ 2497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.194:39379: BlockingRecv error: recv got EOF from 127.7.67.194:39379 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 18:30:27 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:27 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:27.965950 20575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:27.960403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.195:43551)
0806 18:30:27.960675 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:27.960689 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:27.960797 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:27.961103 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:27.961106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:27.961303 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:27.961464 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:27.961474 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:27.962428 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:27.962431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:27.963110 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:27.963121 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:27.963536 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:27.963539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:27.963600 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:27.964073 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:27.964085 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:27.965814 (+ 1729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":79,"threads_started":1}
W20260806 18:30:27.966102 20572 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:27.960504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:43551 (local address 127.7.67.254:34483)
0806 18:30:27.960676 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:27.960680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:27.960756 (+ 76us) server_negotiation.cc:407] Connection header received
0806 18:30:27.960925 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:27.960928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:27.960996 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:27.961090 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:27.961571 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:27.962293 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:27.963213 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:27.963439 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:27.963520 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:27.965970 (+ 2450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.195:43551: BlockingRecv error: recv got EOF from 127.7.67.195:43551 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 18:30:28 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:28 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:28.589282 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:28.583085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.193:45499)
0806 18:30:28.583353 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:28.583370 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:28.583487 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:28.583740 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:28.583743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:28.583962 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:28.584089 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:28.584098 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:28.585058 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:28.585062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:28.585768 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:28.585781 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:28.586226 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:28.586230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:28.586311 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:28.586887 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:28.586900 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:28.589079 (+ 2179us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":77,"threads_started":1}
W20260806 18:30:28.589512 20583 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:28.583177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:45499 (local address 127.7.67.254:34483)
0806 18:30:28.583485 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:28.583489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:28.583505 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:30:28.583576 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:28.583579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:28.583627 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:28.583702 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:28.584230 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:28.584953 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:28.585884 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:28.586123 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:28.586208 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:28.589240 (+ 3032us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.193:45499: BlockingRecv error: recv got EOF from 127.7.67.193:45499 (error 108)
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":76,"threads_started":1}
Aug 06 18:30:28 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:28 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:28.776633 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:28.771380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.194:57529)
0806 18:30:28.771639 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:28.771653 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:28.771752 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:28.772003 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:28.772006 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:28.772203 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:28.772305 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:28.772311 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:28.773058 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:28.773061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:28.773636 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:28.773645 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:28.774006 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:28.774009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:28.774065 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:28.774589 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:28.774604 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:28.776490 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":91,"threads_started":1}
W20260806 18:30:28.776736 20583 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:28.771445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:57529 (local address 127.7.67.254:34483)
0806 18:30:28.771586 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:28.771591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:28.771715 (+ 124us) server_negotiation.cc:407] Connection header received
0806 18:30:28.771821 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:28.771823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:28.771870 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:28.771937 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:28.772426 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:28.772938 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:28.773725 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:28.773931 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:28.773996 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:28.776628 (+ 2632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.194:57529: BlockingRecv error: recv got EOF from 127.7.67.194:57529 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 18:30:28 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:28 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:28.972297 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:28.966779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.195:38311)
0806 18:30:28.967088 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:28.967102 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:28.967204 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:28.967436 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:28.967439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:28.967649 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:28.967746 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:28.967752 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:28.968527 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:28.968531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:28.969083 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:28.969090 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:28.969546 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:28.969550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:28.969631 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:28.970116 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:28.970126 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:28.972143 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":126,"threads_started":1}
W20260806 18:30:28.972383 20583 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:28.966900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:38311 (local address 127.7.67.254:34483)
0806 18:30:28.967049 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:28.967054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:28.967164 (+ 110us) server_negotiation.cc:407] Connection header received
0806 18:30:28.967274 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:28.967276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:28.967324 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:28.967396 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:28.967858 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:28.968417 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:28.969192 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:28.969418 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:28.969505 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:28.972282 (+ 2777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.195:38311: BlockingRecv error: recv got EOF from 127.7.67.195:38311 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 18:30:29 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:29 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:29.595752 20588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:29.590204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.193:57245)
0806 18:30:29.590495 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:29.590523 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:29.590614 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:29.590865 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:29.590868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:29.591100 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:29.591212 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:29.591219 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:29.591956 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:29.591958 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:29.592534 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:29.592541 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:29.592955 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:29.592958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:29.593015 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:29.593513 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:29.593525 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:29.595625 (+ 2100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":90,"threads_started":1}
W20260806 18:30:29.595865 20590 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:29.590307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:57245 (local address 127.7.67.254:34483)
0806 18:30:29.590563 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:29.590566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:29.590581 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:30:29.590691 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:29.590693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:29.590738 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:29.590821 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:29.591314 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:29.591869 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:29.592636 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:29.592868 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:29.592945 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:29.595749 (+ 2804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.193:57245: BlockingRecv error: recv got EOF from 127.7.67.193:57245 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":85,"threads_started":1}
Aug 06 18:30:29 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:29 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:29.783243 20592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:29.777535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.194:36051)
0806 18:30:29.777805 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:29.777818 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:29.777932 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:29.778179 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:29.778182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:29.778387 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:29.778494 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:29.778500 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:29.779356 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:29.779360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:29.779925 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:29.779933 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:29.780371 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:29.780374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:29.780459 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:29.780996 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:29.781008 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:29.783086 (+ 2078us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":97,"threads_started":1}
W20260806 18:30:29.783311 20590 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:29.777589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:36051 (local address 127.7.67.254:34483)
0806 18:30:29.777753 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:29.777758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:29.777890 (+ 132us) server_negotiation.cc:407] Connection header received
0806 18:30:29.778025 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:29.778029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:29.778077 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:29.778142 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:29.778614 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:29.779246 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:29.780042 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:29.780270 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:29.780357 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:29.783210 (+ 2853us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.194:36051: BlockingRecv error: recv got EOF from 127.7.67.194:36051 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 18:30:29 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:29 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:29.978276 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:29.973113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.195:52931)
0806 18:30:29.973381 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:29.973394 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:29.973512 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:29.973734 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:29.973735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:29.973928 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:29.974030 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:29.974035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:29.974703 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:29.974705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:29.975320 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:29.975329 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:29.975793 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:29.975796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:29.975865 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:29.976341 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:29.976353 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:29.978152 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":77,"threads_started":1}
W20260806 18:30:29.978405 20590 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:29.973248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:52931 (local address 127.7.67.254:34483)
0806 18:30:29.973383 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:29.973387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:29.973497 (+ 110us) server_negotiation.cc:407] Connection header received
0806 18:30:29.973596 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:29.973599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:29.973649 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:29.973709 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:29.974117 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:29.974626 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:29.975434 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:29.975668 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:29.975753 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:29.978294 (+ 2541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.195:52931: BlockingRecv error: recv got EOF from 127.7.67.195:52931 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 18:30:30 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:30 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:30.602074 20595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:30.596531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.193:60409)
0806 18:30:30.596840 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:30.596855 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:30.596966 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:30.597192 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:30.597194 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:30.597424 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:30.597533 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:30.597539 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:30.598323 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:30.598327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:30.598990 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:30.599001 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:30.599525 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:30.599529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:30.599598 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:30.600116 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:30.600128 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:30.601945 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":94,"threads_started":1}
W20260806 18:30:30.602226 20596 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:30.596598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:60409 (local address 127.7.67.254:34483)
0806 18:30:30.596840 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:30.596844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:30.596926 (+ 82us) server_negotiation.cc:407] Connection header received
0806 18:30:30.597039 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:30.597041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:30.597088 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:30.597164 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:30.597653 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:30.598210 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:30.599116 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:30.599368 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:30.599476 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:30.602096 (+ 2620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.193:60409: BlockingRecv error: recv got EOF from 127.7.67.193:60409 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":58,"threads_started":1}
Aug 06 18:30:30 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:30 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:30.789324 20597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:30.784020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.194:51477)
0806 18:30:30.784304 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:30.784318 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:30.784423 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:30.784649 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:30.784651 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:30.784834 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:30.784987 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:30.784994 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:30.785765 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:30.785767 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:30.786319 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:30.786325 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:30.786734 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:30.786737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:30.786818 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:30.787386 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:30.787399 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:30.789199 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":78,"threads_started":1}
W20260806 18:30:30.789465 20596 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:30.784112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:51477 (local address 127.7.67.254:34483)
0806 18:30:30.784274 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:30.784279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:30.784383 (+ 104us) server_negotiation.cc:407] Connection header received
0806 18:30:30.784511 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:30.784513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:30.784560 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:30.784618 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:30.785124 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:30.785676 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:30.786404 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:30.786615 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:30.786685 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:30.789340 (+ 2655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.194:51477: BlockingRecv error: recv got EOF from 127.7.67.194:51477 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 18:30:30 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:30 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:30.984599 20599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:30.979114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.195:56529)
0806 18:30:30.979441 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:30.979455 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:30.979575 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:30.979848 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:30.979850 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:30.980084 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:30.980191 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:30.980198 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:30.980981 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:30.980984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:30.981540 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:30.981548 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:30.981994 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:30.981998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:30.982080 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:30.982625 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:30.982637 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:30.984441 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":73,"threads_started":1}
W20260806 18:30:30.984692 20596 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:30.979206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:56529 (local address 127.7.67.254:34483)
0806 18:30:30.979387 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:30.979392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:30.979528 (+ 136us) server_negotiation.cc:407] Connection header received
0806 18:30:30.979667 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:30.979670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:30.979731 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:30.979817 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:30.980311 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:30.980869 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:30.981652 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:30.981878 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:30.981963 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:30.984585 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.195:56529: BlockingRecv error: recv got EOF from 127.7.67.195:56529 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 18:30:31 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:31 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:31.608073 20600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:31.602917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.193:41787)
0806 18:30:31.603208 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:31.603221 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:31.603321 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:31.603565 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:31.603567 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:31.603756 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:31.603862 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:31.603866 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:31.604581 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:31.604583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:31.605141 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:31.605148 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:31.605507 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:31.605510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:31.605567 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:31.606039 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:31.606050 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:31.607933 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":93,"threads_started":1}
W20260806 18:30:31.608214 20601 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:31.602961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:41787 (local address 127.7.67.254:34483)
0806 18:30:31.603227 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:31.603231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:31.603286 (+ 55us) server_negotiation.cc:407] Connection header received
0806 18:30:31.603393 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:31.603395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:31.603441 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:31.603522 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:31.603956 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:31.604493 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:31.605223 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:31.605432 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:31.605498 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:31.608086 (+ 2588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.193:41787: BlockingRecv error: recv got EOF from 127.7.67.193:41787 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":75,"threads_started":1}
Aug 06 18:30:31 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:31 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:31.795672 20602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:31.790170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.194:48927)
0806 18:30:31.790461 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:31.790476 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:31.790604 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:31.790872 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:31.790875 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:31.791088 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:31.791192 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:31.791198 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:31.791937 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:31.791941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:31.792507 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:31.792515 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:31.792962 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:31.792965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:31.793045 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:31.793594 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:31.793609 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:31.795549 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":90,"threads_started":1}
W20260806 18:30:31.795809 20601 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:31.790273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:48927 (local address 127.7.67.254:34483)
0806 18:30:31.790421 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:31.790427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:31.790553 (+ 126us) server_negotiation.cc:407] Connection header received
0806 18:30:31.790689 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:31.790692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:31.790741 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:31.790809 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:31.791305 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:31.791832 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:31.792615 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:31.792858 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:31.792944 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:31.795686 (+ 2742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.194:48927: BlockingRecv error: recv got EOF from 127.7.67.194:48927 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 18:30:31 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:31 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:32.003536 20604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:31.985495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.195:49183)
0806 18:30:31.985794 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:31.985808 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:31.985919 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:31.986138 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:31.986140 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:31.986328 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:31.986427 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:31.986432 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:31.987282 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:31.987284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:31.987846 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:31.987855 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:31.988265 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:31.988267 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:31.988324 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:32.001152 (+ 12828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:32.001169 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:32.003350 (+ 2181us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":91,"threads_started":1}
W20260806 18:30:32.003659 20601 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:31.985632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:49183 (local address 127.7.67.254:34483)
0806 18:30:31.985811 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:31.985815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:31.985878 (+ 63us) server_negotiation.cc:407] Connection header received
0806 18:30:31.985992 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:31.985994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:31.986043 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:31.986111 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:31.986574 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:31.987179 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:31.987958 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:31.988176 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:31.988256 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:32.003533 (+ 15277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.195:49183: BlockingRecv error: recv got EOF from 127.7.67.195:49183 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 06 18:30:32 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:32 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:32.613945 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:32.608899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.193:38385)
0806 18:30:32.609179 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:32.609192 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:32.609295 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:32.609507 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:32.609510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:32.609703 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:32.609809 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:32.609815 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:32.610509 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:32.610511 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:32.611127 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:32.611135 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:32.611519 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:32.611522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:32.611590 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:32.612047 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:32.612059 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:32.613807 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":74,"threads_started":1}
W20260806 18:30:32.614055 20608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:32.608954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:38385 (local address 127.7.67.254:34483)
0806 18:30:32.609187 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:32.609191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:32.609254 (+ 63us) server_negotiation.cc:407] Connection header received
0806 18:30:32.609363 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:32.609365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:32.609409 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:32.609485 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:32.609893 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:32.610433 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:32.611215 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:32.611437 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:32.611512 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:32.613912 (+ 2400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.193:38385: BlockingRecv error: recv got EOF from 127.7.67.193:38385 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":73,"threads_started":1}
Aug 06 18:30:32 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:32 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:32.801908 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:32.796485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.194:44603)
0806 18:30:32.796770 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:32.796783 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:32.796883 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:32.797147 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:32.797150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:32.797358 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:32.797471 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:32.797478 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:32.798187 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:32.798190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:32.798794 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:32.798811 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:32.799304 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:32.799307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:32.799380 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:32.799930 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:32.799950 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:32.801759 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":94,"threads_started":1}
W20260806 18:30:32.802371 20608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:32.796627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:44603 (local address 127.7.67.254:34483)
0806 18:30:32.796775 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:32.796778 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:32.796845 (+ 67us) server_negotiation.cc:407] Connection header received
0806 18:30:32.796952 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:32.796954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:32.796999 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:32.797069 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:32.797583 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:32.798099 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:32.798973 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:32.799204 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:32.799286 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:32.801946 (+ 2660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.194:44603: BlockingRecv error: recv got EOF from 127.7.67.194:44603 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 18:30:33 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:33 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:33.009917 20610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:33.004376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.195:36651)
0806 18:30:33.004703 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:33.004718 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:33.004845 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:33.005217 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:33.005219 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:33.005419 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:33.005521 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:33.005527 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:33.006244 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:33.006247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:33.006903 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:33.006914 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:33.007336 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:33.007341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:33.007417 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:33.007958 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:33.007971 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:33.009763 (+ 1792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":95,"threads_started":1}
W20260806 18:30:33.010012 20608 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:33.004493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:36651 (local address 127.7.67.254:34483)
0806 18:30:33.004665 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:33.004669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:33.004790 (+ 121us) server_negotiation.cc:407] Connection header received
0806 18:30:33.004995 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:33.004998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:33.005067 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:33.005142 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:33.005617 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:33.006141 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:33.006993 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:33.007209 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:33.007278 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:33.009904 (+ 2626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.195:36651: BlockingRecv error: recv got EOF from 127.7.67.195:36651 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 18:30:33 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:33 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:33.620494 20611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:33.614944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.193:44509)
0806 18:30:33.615230 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:33.615243 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:33.615339 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:33.615599 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:33.615601 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:33.615789 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:33.615906 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:33.615912 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:33.616634 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:33.616636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:33.617226 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:33.617235 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:33.617754 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:33.617759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:33.617850 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:33.618431 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:33.618446 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:33.620362 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":84,"threads_started":1}
W20260806 18:30:33.620620 20612 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:33.615065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:44509 (local address 127.7.67.254:34483)
0806 18:30:33.615377 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:33.615381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:33.615396 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:30:33.615439 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:33.615441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:33.615483 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:33.615574 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:33.615999 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:33.616538 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:33.617354 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:33.617597 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:33.617704 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:33.620497 (+ 2793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.193:44509: BlockingRecv error: recv got EOF from 127.7.67.193:44509 (error 108)
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":74,"threads_started":1}
Aug 06 18:30:33 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:33 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:33.808233 20614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:33.802784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.194:44609)
0806 18:30:33.803130 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:33.803147 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:33.803269 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:33.803511 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:33.803514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:33.803729 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:33.803856 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:33.803865 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:33.804619 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:33.804622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:33.805199 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:33.805207 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:33.805633 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:33.805637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:33.805713 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:33.806181 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:33.806192 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:33.808094 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":124,"threads_started":1}
W20260806 18:30:33.808385 20612 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:33.802914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:44609 (local address 127.7.67.254:34483)
0806 18:30:33.803106 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:33.803111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:33.803220 (+ 109us) server_negotiation.cc:407] Connection header received
0806 18:30:33.803356 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:33.803359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:33.803409 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:33.803475 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:33.803965 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:33.804512 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:33.805310 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:33.805534 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:33.805618 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:33.808258 (+ 2640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.194:44609: BlockingRecv error: recv got EOF from 127.7.67.194:44609 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 06 18:30:34 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:34 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.195@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:34.016386 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:34.010768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.195:56651)
0806 18:30:34.011113 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:34.011128 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:34.011247 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:34.011485 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:34.011487 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:34.011689 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:34.011797 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:34.011804 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:34.012501 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:34.012504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:34.013087 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:34.013095 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:34.013536 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:34.013539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:34.013611 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:34.014100 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:34.014112 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:34.016214 (+ 2102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":134,"threads_started":1}
W20260806 18:30:34.016441 20612 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:34.010921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:56651 (local address 127.7.67.254:34483)
0806 18:30:34.011059 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:34.011062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:34.011197 (+ 135us) server_negotiation.cc:407] Connection header received
0806 18:30:34.011339 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:34.011341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:34.011390 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:34.011450 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:34.011889 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:34.012414 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:34.013200 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:34.013424 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:34.013498 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:34.016340 (+ 2842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.195:56651: BlockingRecv error: recv got EOF from 127.7.67.195:56651 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 18:30:34 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:34 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.193@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:34.626842 20616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:34.621407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.193:53363)
0806 18:30:34.621701 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:34.621716 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:34.621816 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:34.622043 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:34.622045 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:34.622235 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:34.622336 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:34.622342 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:34.623085 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:34.623088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:34.623678 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:34.623687 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:34.624065 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:34.624067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:34.624124 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:34.624609 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:34.624620 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:34.626662 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":81,"threads_started":1}
W20260806 18:30:34.626989 20617 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:34.621488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:53363 (local address 127.7.67.254:34483)
0806 18:30:34.621736 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:34.621740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:34.621769 (+ 29us) server_negotiation.cc:407] Connection header received
0806 18:30:34.621891 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:34.621893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:34.621935 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:34.622027 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:34.622422 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:34.622980 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:34.623766 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:34.623980 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:34.624054 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:34.626820 (+ 2766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.193:53363: BlockingRecv error: recv got EOF from 127.7.67.193:53363 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":68,"threads_started":1}
Aug 06 18:30:34 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:30:34 dist-test-slave-5bn2 krb5kdc[19163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.194@KRBTEST.COM for oryx/127.7.67.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:30:34.814414 20618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:34.809163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34483 (local address 127.7.67.194:33099)
0806 18:30:34.809429 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:34.809441 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:34.809548 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:34.809766 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:34.809768 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:34.809960 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:34.810065 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:34.810083 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:34.810762 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:34.810764 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:34.811362 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:34.811370 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:34.811750 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:34.811752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:34.811809 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:34.812272 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:34.812282 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:34.814245 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.254:34483: Server oryx/127.7.67.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":75,"threads_started":1}
W20260806 18:30:34.814486 20617 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:34.809263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:33099 (local address 127.7.67.254:34483)
0806 18:30:34.809397 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:34.809401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:34.809502 (+ 101us) server_negotiation.cc:407] Connection header received
0806 18:30:34.809622 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:34.809624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:34.809669 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:34.809743 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:34.810166 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:34.810684 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:34.811453 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:34.811670 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:34.811739 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:34.814381 (+ 2642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.194:33099: BlockingRecv error: recv got EOF from 127.7.67.194:33099 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:30:34.893953 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 19958
I20260806 18:30:34.908102 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 20169
I20260806 18:30:34.923065 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 20357
I20260806 18:30:34.932502 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 19859
2026-08-06T18:30:34Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18113 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T18:30:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:30:34Z Disabled control of system clock
I20260806 18:30:34.967370 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:44775
--webserver_interface=127.7.67.254
--webserver_port=0
--builtin_ntp_servers=127.7.67.212:42547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:44775
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 18:30:35.080556 20626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:35.080837 20626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:35.080870 20626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:35.084923 20626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:30:35.084987 20626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:35.085011 20626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:35.085032 20626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:30:35.085053 20626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:30:35.089769 20626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:44775
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:44775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:35.090821 20626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:35.092218 20626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:35.100708 20636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:35.100703 20640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:35.100703 20633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:35.100930 20626 server_base.cc:1034] running on GCE node
I20260806 18:30:35.101583 20626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:35.102676 20626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:35.103834 20626 hybrid_clock.cc:648] HybridClock initialized: now 1786041035103818 us; error 43 us; skew 500 ppm
I20260806 18:30:35.105679 20626 webserver.cc:460] Webserver started at http://127.7.67.254:33551/ using document root <none> and password file <none>
I20260806 18:30:35.106251 20626 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:35.106315 20626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:35.106527 20626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:35.108634 20626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "8bfa8bdd404249f889155ce195c64082"
format_stamp: "Formatted at 2026-08-06 18:30:35 on dist-test-slave-5bn2"
server_key: "8785f7fdda6203d367d259f6a380e8f8"
server_key_iv: "c46a94d7e8187bb86a6616a243d2d02c"
server_key_version: "encryptionkey@0"
I20260806 18:30:35.109233 20626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "8bfa8bdd404249f889155ce195c64082"
format_stamp: "Formatted at 2026-08-06 18:30:35 on dist-test-slave-5bn2"
server_key: "8785f7fdda6203d367d259f6a380e8f8"
server_key_iv: "c46a94d7e8187bb86a6616a243d2d02c"
server_key_version: "encryptionkey@0"
I20260806 18:30:35.113237 20626 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:30:35.115685 20648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:35.116878 20626 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:30:35.116998 20626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "8bfa8bdd404249f889155ce195c64082"
format_stamp: "Formatted at 2026-08-06 18:30:35 on dist-test-slave-5bn2"
server_key: "8785f7fdda6203d367d259f6a380e8f8"
server_key_iv: "c46a94d7e8187bb86a6616a243d2d02c"
server_key_version: "encryptionkey@0"
I20260806 18:30:35.117106 20626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:35.132707 20626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:35.133459 20626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:35.133658 20626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:35.140506 20626 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:44775
I20260806 18:30:35.140529 20713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:44775 every 8 connection(s)
I20260806 18:30:35.141453 20626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:30:35.143323 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 20626
I20260806 18:30:35.143445 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:30:35.143779 7439 external_mini_cluster.cc:1428] Setting key adafddd7f04829f94df873dc89aac2d2
I20260806 18:30:35.144657 20714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:35.150764 20714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082: Bootstrap starting.
I20260806 18:30:35.153434 20720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:35.145212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57282 (local address 127.7.67.254:44775)
0806 18:30:35.145853 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:35.145867 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:35.145902 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:30:35.146559 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:35.146575 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:35.146996 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:35.147293 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:35.147713 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:35.148666 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:35.149664 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:35.150362 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:35.150543 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:35.151422 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:35.151428 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:35.151445 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:35.151933 (+ 488us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:35.152011 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:35.152040 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:35.152139 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:35.152230 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 18:30:35.152511 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":146,"threads_started":1}
I20260806 18:30:35.153580 20714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:35.154335 20714 log.cc:824] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:35.156320 20714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082: No bootstrap required, opened a new log
I20260806 18:30:35.159801 20714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bfa8bdd404249f889155ce195c64082" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44775 } }
I20260806 18:30:35.160040 20714 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:35.160091 20714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bfa8bdd404249f889155ce195c64082, State: Initialized, Role: FOLLOWER
I20260806 18:30:35.160544 20714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bfa8bdd404249f889155ce195c64082" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44775 } }
I20260806 18:30:35.160712 20714 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:30:35.160781 20714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:30:35.160877 20714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:35.161770 20714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bfa8bdd404249f889155ce195c64082" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44775 } }
I20260806 18:30:35.162104 20714 leader_election.cc:302] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bfa8bdd404249f889155ce195c64082; no voters:
I20260806 18:30:35.162396 20714 leader_election.cc:288] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:30:35.162513 20723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:35.162703 20723 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082 [term 1 LEADER]: Becoming Leader. State: Replica: 8bfa8bdd404249f889155ce195c64082, State: Running, Role: LEADER
I20260806 18:30:35.163076 20723 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bfa8bdd404249f889155ce195c64082" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44775 } }
I20260806 18:30:35.163605 20714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:30:35.164728 20724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bfa8bdd404249f889155ce195c64082" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bfa8bdd404249f889155ce195c64082" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44775 } } }
I20260806 18:30:35.164758 20725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bfa8bdd404249f889155ce195c64082. Latest consensus state: current_term: 1 leader_uuid: "8bfa8bdd404249f889155ce195c64082" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bfa8bdd404249f889155ce195c64082" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44775 } } }
I20260806 18:30:35.164963 20724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:35.164960 20725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:35.165405 20737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:30:35.168309 20737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:30:35.173635 20737 catalog_manager.cc:1347] Generated new cluster ID: ef50eac869cf4249b7c3fe81cc8762de
I20260806 18:30:35.173709 20737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:30:35.187253 20737 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:30:35.188187 20737 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:30:35.192235 20737 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bfa8bdd404249f889155ce195c64082: Generated new TSK 0
I20260806 18:30:35.192965 20737 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:30:35.204049 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:44775
--builtin_ntp_servers=127.7.67.212:42547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 18:30:35.319345 20751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:35.319633 20751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:35.319674 20751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:35.323802 20751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:35.323872 20751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:35.323927 20751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:30:35.328728 20751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:44775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:35.329897 20751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:35.331410 20751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:35.341647 20757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:35.341723 20751 server_base.cc:1034] running on GCE node
W20260806 18:30:35.341655 20761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:35.341655 20758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:35.342347 20751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:35.343036 20751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:35.344187 20751 hybrid_clock.cc:648] HybridClock initialized: now 1786041035344160 us; error 40 us; skew 500 ppm
I20260806 18:30:35.345970 20751 webserver.cc:460] Webserver started at http://127.7.67.193:35479/ using document root <none> and password file <none>
I20260806 18:30:35.346545 20751 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:35.346629 20751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:35.346916 20751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:35.349228 20751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "09fbce77f6a14d2597a9d63a50e1f21c"
format_stamp: "Formatted at 2026-08-06 18:30:35 on dist-test-slave-5bn2"
server_key: "efa9e87cfb666726326ba22e07b39827"
server_key_iv: "e8c9b801dfa0f0ed228218d598109f50"
server_key_version: "encryptionkey@0"
I20260806 18:30:35.349800 20751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "09fbce77f6a14d2597a9d63a50e1f21c"
format_stamp: "Formatted at 2026-08-06 18:30:35 on dist-test-slave-5bn2"
server_key: "efa9e87cfb666726326ba22e07b39827"
server_key_iv: "e8c9b801dfa0f0ed228218d598109f50"
server_key_version: "encryptionkey@0"
I20260806 18:30:35.353492 20751 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260806 18:30:35.356058 20769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:35.357257 20751 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:30:35.357369 20751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "09fbce77f6a14d2597a9d63a50e1f21c"
format_stamp: "Formatted at 2026-08-06 18:30:35 on dist-test-slave-5bn2"
server_key: "efa9e87cfb666726326ba22e07b39827"
server_key_iv: "e8c9b801dfa0f0ed228218d598109f50"
server_key_version: "encryptionkey@0"
I20260806 18:30:35.357471 20751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:35.389232 20751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:35.390020 20751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:35.390225 20751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:35.390839 20751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:35.391887 20751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:35.391938 20751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:35.391993 20751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:35.392019 20751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:35.400704 20751 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:35739
I20260806 18:30:35.400787 20952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:35739 every 8 connection(s)
I20260806 18:30:35.401823 20751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:30:35.411238 20720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:35.403756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:35703 (local address 127.7.67.254:44775)
0806 18:30:35.403920 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:35.403925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:35.405100 (+ 1175us) server_negotiation.cc:407] Connection header received
0806 18:30:35.406537 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:35.406540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:35.406597 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:35.406687 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:35.407695 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:35.408237 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:35.409358 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:35.409596 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:35.409676 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:35.410710 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:35.410714 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:35.410716 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:35.410728 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:35.410749 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:35.410758 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:35.410817 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:35.411039 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 18:30:35.411112 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:30:35.411926 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 20751
I20260806 18:30:35.412065 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:30:35.412146 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:35.404161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44775 (local address 127.7.67.193:35703)
0806 18:30:35.404936 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:35.404994 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:35.406311 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:35.406927 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:35.406935 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:35.407023 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:35.407541 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:35.407556 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:35.408329 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:35.408331 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:35.409242 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:35.409252 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:35.409721 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:35.409723 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:35.409861 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:35.410448 (+ 587us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:30:35.410450 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:35.410490 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:30:35.410563 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:35.410822 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:35.410827 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:35.410830 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:30:35.410949 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 18:30:35.411184 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":121,"threads_started":1}
I20260806 18:30:35.412413 7439 external_mini_cluster.cc:1428] Setting key c583c256d14c4d0c184188042d99b20d
I20260806 18:30:35.413455 20953 heartbeater.cc:347] Connected to a master server at 127.7.67.254:44775
I20260806 18:30:35.413713 20953 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:35.414346 20953 heartbeater.cc:511] Master 127.7.67.254:44775 requested a full tablet report, sending...
I20260806 18:30:35.414511 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:44775
--builtin_ntp_servers=127.7.67.212:42547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 18:30:35.415977 20669 ts_manager.cc:194] Registered new tserver with Master: 09fbce77f6a14d2597a9d63a50e1f21c (127.7.67.193:35739)
I20260806 18:30:35.417132 20669 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.193:35703
W20260806 18:30:35.528354 20958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:35.528640 20958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:35.528677 20958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:35.532768 20958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:35.532831 20958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:35.532884 20958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:30:35.537679 20958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:44775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:35.538800 20958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:35.540246 20958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:35.549737 20963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:35.549732 20966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:35.549911 20958 server_base.cc:1034] running on GCE node
W20260806 18:30:35.549732 20964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:35.550457 20958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:35.551146 20958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:35.552341 20958 hybrid_clock.cc:648] HybridClock initialized: now 1786041035552314 us; error 34 us; skew 500 ppm
I20260806 18:30:35.554221 20958 webserver.cc:460] Webserver started at http://127.7.67.194:34435/ using document root <none> and password file <none>
I20260806 18:30:35.554805 20958 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:35.554910 20958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:35.555132 20958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:35.557200 20958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "67e559222f7c41379f7b1289f43dd7f7"
format_stamp: "Formatted at 2026-08-06 18:30:35 on dist-test-slave-5bn2"
server_key: "eebabeb745becf044c7be5194f2511a1"
server_key_iv: "3f62f6a7c3e1addccbcd1b0bf3d0992c"
server_key_version: "encryptionkey@0"
I20260806 18:30:35.557792 20958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "67e559222f7c41379f7b1289f43dd7f7"
format_stamp: "Formatted at 2026-08-06 18:30:35 on dist-test-slave-5bn2"
server_key: "eebabeb745becf044c7be5194f2511a1"
server_key_iv: "3f62f6a7c3e1addccbcd1b0bf3d0992c"
server_key_version: "encryptionkey@0"
I20260806 18:30:35.561681 20958 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260806 18:30:35.564318 20974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:35.565619 20958 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:30:35.565732 20958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "67e559222f7c41379f7b1289f43dd7f7"
format_stamp: "Formatted at 2026-08-06 18:30:35 on dist-test-slave-5bn2"
server_key: "eebabeb745becf044c7be5194f2511a1"
server_key_iv: "3f62f6a7c3e1addccbcd1b0bf3d0992c"
server_key_version: "encryptionkey@0"
I20260806 18:30:35.565840 20958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:35.587613 20958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:35.588392 20958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:35.588590 20958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:35.589216 20958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:35.590252 20958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:35.590304 20958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:35.590361 20958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:35.590396 20958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:35.599177 20958 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:39715
I20260806 18:30:35.599218 21135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:39715 every 8 connection(s)
I20260806 18:30:35.600136 20958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:30:35.602365 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 20958
I20260806 18:30:35.602510 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:30:35.602898 7439 external_mini_cluster.cc:1428] Setting key c490949d6f94e52e6651cf33650f3b8b
I20260806 18:30:35.604981 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:44775
--builtin_ntp_servers=127.7.67.212:42547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 18:30:35.610590 20720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:35.602050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:49623 (local address 127.7.67.254:44775)
0806 18:30:35.602202 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:35.602205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:35.603538 (+ 1333us) server_negotiation.cc:407] Connection header received
0806 18:30:35.605142 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:35.605146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:35.605202 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:35.605283 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:35.606200 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:35.606917 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:35.608540 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:35.608861 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:35.608974 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:35.610061 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:35.610065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:35.610080 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:35.610097 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:35.610123 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:35.610134 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:35.610213 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:35.610333 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 18:30:35.610422 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:30:35.611470 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:35.602575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44775 (local address 127.7.67.194:49623)
0806 18:30:35.603373 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:35.603418 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:35.604914 (+ 1496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:35.605460 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:35.605467 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:35.605556 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:35.606060 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:35.606073 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:35.607061 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:35.607065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:35.608391 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:35.608402 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:35.609016 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:35.609019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:35.609171 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:35.609777 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:30:35.609779 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:35.609822 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:30:35.609896 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:35.610212 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:35.610217 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:35.610221 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:30:35.610313 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 18:30:35.610560 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":167,"threads_started":1}
I20260806 18:30:35.612692 21139 heartbeater.cc:347] Connected to a master server at 127.7.67.254:44775
I20260806 18:30:35.612946 21139 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:35.613497 21139 heartbeater.cc:511] Master 127.7.67.254:44775 requested a full tablet report, sending...
I20260806 18:30:35.614625 20669 ts_manager.cc:194] Registered new tserver with Master: 67e559222f7c41379f7b1289f43dd7f7 (127.7.67.194:39715)
I20260806 18:30:35.615294 20669 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.194:49623
W20260806 18:30:35.721818 21143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:35.722102 21143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:35.722141 21143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:35.726137 21143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:35.726204 21143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:35.726271 21143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:30:35.731019 21143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:44775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:35.732133 21143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:35.733526 21143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:35.742916 21148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:35.742903 21149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:35.742903 21151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:35.743251 21143 server_base.cc:1034] running on GCE node
I20260806 18:30:35.743666 21143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:35.744266 21143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:35.745450 21143 hybrid_clock.cc:648] HybridClock initialized: now 1786041035745421 us; error 40 us; skew 500 ppm
I20260806 18:30:35.747342 21143 webserver.cc:460] Webserver started at http://127.7.67.195:45399/ using document root <none> and password file <none>
I20260806 18:30:35.747920 21143 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:35.747985 21143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:35.748195 21143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:35.750294 21143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "d2f9ed5778564b99a940ab5447ccc1bc"
format_stamp: "Formatted at 2026-08-06 18:30:35 on dist-test-slave-5bn2"
server_key: "071d7e0de3d4ad1504da342787d4700f"
server_key_iv: "a9a12e1665fff29a5e32f13eb63045cf"
server_key_version: "encryptionkey@0"
I20260806 18:30:35.750886 21143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "d2f9ed5778564b99a940ab5447ccc1bc"
format_stamp: "Formatted at 2026-08-06 18:30:35 on dist-test-slave-5bn2"
server_key: "071d7e0de3d4ad1504da342787d4700f"
server_key_iv: "a9a12e1665fff29a5e32f13eb63045cf"
server_key_version: "encryptionkey@0"
I20260806 18:30:35.754518 21143 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:30:35.757067 21158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:35.758316 21143 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:30:35.758446 21143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "d2f9ed5778564b99a940ab5447ccc1bc"
format_stamp: "Formatted at 2026-08-06 18:30:35 on dist-test-slave-5bn2"
server_key: "071d7e0de3d4ad1504da342787d4700f"
server_key_iv: "a9a12e1665fff29a5e32f13eb63045cf"
server_key_version: "encryptionkey@0"
I20260806 18:30:35.758555 21143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:35.774936 21143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:35.775650 21143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:35.775831 21143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:35.776432 21143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:35.777384 21143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:35.777443 21143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:35.777499 21143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:35.777526 21143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:35.785997 21143 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:37559
I20260806 18:30:35.786034 21327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:37559 every 8 connection(s)
I20260806 18:30:35.786955 21143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:30:35.792852 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 21143
I20260806 18:30:35.792961 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:30:35.793239 7439 external_mini_cluster.cc:1428] Setting key 2d375427c9fe873f2ef01e0dadfe5a25
I20260806 18:30:35.796067 20720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:35.788748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:40163 (local address 127.7.67.254:44775)
0806 18:30:35.788875 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:35.788879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:35.789936 (+ 1057us) server_negotiation.cc:407] Connection header received
0806 18:30:35.791342 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:35.791345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:35.791397 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:35.791473 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:35.792356 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:35.792865 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:35.794219 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:35.794469 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:35.794585 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:35.795695 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:35.795698 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:35.795700 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:35.795712 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:35.795733 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:35.795741 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:35.795797 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:35.795901 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 18:30:35.795954 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:30:35.796959 21331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:35.789121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44775 (local address 127.7.67.195:40163)
0806 18:30:35.789779 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:35.789820 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:35.791141 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:35.791628 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:35.791635 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:35.791720 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:35.792243 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:35.792255 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:35.793005 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:35.793009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:35.794101 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:35.794114 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:35.794638 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:35.794642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:35.794811 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:35.795443 (+ 632us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:30:35.795445 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:35.795486 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:30:35.795559 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:35.795803 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:35.795808 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:35.795812 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:30:35.795896 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 18:30:35.796125 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":113,"threads_started":1}
I20260806 18:30:35.798046 21328 heartbeater.cc:347] Connected to a master server at 127.7.67.254:44775
I20260806 18:30:35.798275 21328 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:35.798803 21328 heartbeater.cc:511] Master 127.7.67.254:44775 requested a full tablet report, sending...
I20260806 18:30:35.799819 20669 ts_manager.cc:194] Registered new tserver with Master: d2f9ed5778564b99a940ab5447ccc1bc (127.7.67.195:37559)
I20260806 18:30:35.800376 20669 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.195:40163
I20260806 18:30:35.807319 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 18:30:35.810305 20720 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:35.809411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57286 (local address 127.7.67.254:44775)
0806 18:30:35.809533 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:35.809537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:35.809747 (+ 210us) server_negotiation.cc:407] Connection header received
0806 18:30:35.809872 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:35.809874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:35.809913 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:35.809977 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:35.810196 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57286: BlockingRecv error: recv got EOF from 127.0.0.1:57286 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:30:35.810768 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 20751
I20260806 18:30:35.829502 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 20958
I20260806 18:30:35.845964 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 21143
I20260806 18:30:35.861027 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 20626
2026-08-06T18:30:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (930 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T18:30:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:30:35Z Disabled control of system clock
I20260806 18:30:35.898227 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:46123
--webserver_interface=127.7.67.254
--webserver_port=0
--builtin_ntp_servers=127.7.67.212:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:46123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 18:30:36.009550 21344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:36.009817 21344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:36.009881 21344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:36.013867 21344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:30:36.013935 21344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:36.013955 21344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:36.013975 21344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:30:36.013996 21344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:30:36.018726 21344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:46123
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:46123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:36.019840 21344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:36.021255 21344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:36.029508 21354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:36.029647 21344 server_base.cc:1034] running on GCE node
W20260806 18:30:36.029546 21353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:36.029548 21358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:36.030395 21344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:36.031534 21344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:36.032682 21344 hybrid_clock.cc:648] HybridClock initialized: now 1786041036032671 us; error 27 us; skew 500 ppm
I20260806 18:30:36.034446 21344 webserver.cc:460] Webserver started at http://127.7.67.254:44419/ using document root <none> and password file <none>
I20260806 18:30:36.035048 21344 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:36.035113 21344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:36.035320 21344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:36.037371 21344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "38482a3eb99f468492cb39eb3eddf290"
format_stamp: "Formatted at 2026-08-06 18:30:36 on dist-test-slave-5bn2"
server_key: "d46beafc8b148cf29660b6ffcf93e33f"
server_key_iv: "3c9966a4ab6c7cbb75e862b1b7a3559e"
server_key_version: "encryptionkey@0"
I20260806 18:30:36.037931 21344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "38482a3eb99f468492cb39eb3eddf290"
format_stamp: "Formatted at 2026-08-06 18:30:36 on dist-test-slave-5bn2"
server_key: "d46beafc8b148cf29660b6ffcf93e33f"
server_key_iv: "3c9966a4ab6c7cbb75e862b1b7a3559e"
server_key_version: "encryptionkey@0"
I20260806 18:30:36.041612 21344 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260806 18:30:36.044112 21368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:36.045344 21344 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:30:36.045463 21344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "38482a3eb99f468492cb39eb3eddf290"
format_stamp: "Formatted at 2026-08-06 18:30:36 on dist-test-slave-5bn2"
server_key: "d46beafc8b148cf29660b6ffcf93e33f"
server_key_iv: "3c9966a4ab6c7cbb75e862b1b7a3559e"
server_key_version: "encryptionkey@0"
I20260806 18:30:36.045578 21344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:36.050185 21344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:36.050784 21344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:36.050972 21344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:36.057592 21344 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:46123
I20260806 18:30:36.057600 21440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:46123 every 8 connection(s)
I20260806 18:30:36.058550 21344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:30:36.061811 21441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:36.063315 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 21344
I20260806 18:30:36.063438 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:30:36.063779 7439 external_mini_cluster.cc:1428] Setting key fe41c0d6a13ea6d8bc4a9cd5e5b9c915
I20260806 18:30:36.068378 21441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290: Bootstrap starting.
I20260806 18:30:36.069428 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:36.065334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35840 (local address 127.7.67.254:46123)
0806 18:30:36.066012 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:36.066027 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:36.066065 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:30:36.066714 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:36.066731 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:36.067126 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:36.067432 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:36.068195 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:36.068200 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:36.068211 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:36.068222 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:36.068278 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:36.068294 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:36.068424 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:36.068504 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 18:30:36.068743 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":150,"threads_started":1}
I20260806 18:30:36.070713 21441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:36.071501 21441 log.cc:824] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:36.073362 21441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290: No bootstrap required, opened a new log
I20260806 18:30:36.077073 21441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38482a3eb99f468492cb39eb3eddf290" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 46123 } }
I20260806 18:30:36.077343 21441 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:36.077382 21441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38482a3eb99f468492cb39eb3eddf290, State: Initialized, Role: FOLLOWER
I20260806 18:30:36.077827 21441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38482a3eb99f468492cb39eb3eddf290" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 46123 } }
I20260806 18:30:36.077961 21441 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:30:36.078017 21441 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:30:36.078091 21441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:36.079016 21441 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38482a3eb99f468492cb39eb3eddf290" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 46123 } }
I20260806 18:30:36.079329 21441 leader_election.cc:302] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38482a3eb99f468492cb39eb3eddf290; no voters:
I20260806 18:30:36.079595 21441 leader_election.cc:288] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:30:36.079720 21456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:36.079947 21456 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290 [term 1 LEADER]: Becoming Leader. State: Replica: 38482a3eb99f468492cb39eb3eddf290, State: Running, Role: LEADER
I20260806 18:30:36.080310 21456 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38482a3eb99f468492cb39eb3eddf290" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 46123 } }
I20260806 18:30:36.080819 21441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:30:36.081985 21459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38482a3eb99f468492cb39eb3eddf290. Latest consensus state: current_term: 1 leader_uuid: "38482a3eb99f468492cb39eb3eddf290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38482a3eb99f468492cb39eb3eddf290" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 46123 } } }
I20260806 18:30:36.082131 21459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:36.081992 21458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38482a3eb99f468492cb39eb3eddf290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38482a3eb99f468492cb39eb3eddf290" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 46123 } } }
I20260806 18:30:36.082260 21458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:36.082654 21471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:30:36.085464 21471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:30:36.090777 21471 catalog_manager.cc:1347] Generated new cluster ID: 680aad348cea4da39a845b504488d059
I20260806 18:30:36.090878 21471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:30:36.107592 21471 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:30:36.107944 21471 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:30:36.114369 21471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38482a3eb99f468492cb39eb3eddf290: Generated new TSK 0
I20260806 18:30:36.115142 21471 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:30:36.127475 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:46123
--builtin_ntp_servers=127.7.67.212:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 18:30:36.243394 21488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:36.243688 21488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:36.243747 21488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:36.247785 21488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:36.247857 21488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:36.247943 21488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:30:36.252960 21488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:46123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:36.254180 21488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:36.255668 21488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:30:36.265638 21488 server_base.cc:1034] running on GCE node
W20260806 18:30:36.265594 21497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:36.265597 21502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:36.265594 21496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:36.266194 21488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:36.266928 21488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:36.268085 21488 hybrid_clock.cc:648] HybridClock initialized: now 1786041036268051 us; error 47 us; skew 500 ppm
I20260806 18:30:36.269949 21488 webserver.cc:460] Webserver started at http://127.7.67.193:42185/ using document root <none> and password file <none>
I20260806 18:30:36.270557 21488 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:36.270627 21488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:36.270875 21488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:36.273058 21488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "91bd15068b9247e48b2d776b06e09e7f"
format_stamp: "Formatted at 2026-08-06 18:30:36 on dist-test-slave-5bn2"
server_key: "2fbb8ac693fd2946464c43e052a22efa"
server_key_iv: "b0b85821354ee2e1904a4a76a269bb87"
server_key_version: "encryptionkey@0"
I20260806 18:30:36.273691 21488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "91bd15068b9247e48b2d776b06e09e7f"
format_stamp: "Formatted at 2026-08-06 18:30:36 on dist-test-slave-5bn2"
server_key: "2fbb8ac693fd2946464c43e052a22efa"
server_key_iv: "b0b85821354ee2e1904a4a76a269bb87"
server_key_version: "encryptionkey@0"
I20260806 18:30:36.277582 21488 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 18:30:36.280282 21511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:36.281620 21488 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:30:36.281740 21488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "91bd15068b9247e48b2d776b06e09e7f"
format_stamp: "Formatted at 2026-08-06 18:30:36 on dist-test-slave-5bn2"
server_key: "2fbb8ac693fd2946464c43e052a22efa"
server_key_iv: "b0b85821354ee2e1904a4a76a269bb87"
server_key_version: "encryptionkey@0"
I20260806 18:30:36.281862 21488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:36.286674 21488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:36.287364 21488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:36.287518 21488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:36.288062 21488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:36.289047 21488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:36.289106 21488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:36.289152 21488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:36.289176 21488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:36.298134 21488 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:46285
I20260806 18:30:36.298156 21677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:46285 every 8 connection(s)
I20260806 18:30:36.299191 21488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:30:36.304924 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 21488
I20260806 18:30:36.305056 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:30:36.305348 7439 external_mini_cluster.cc:1428] Setting key 0591a0ecb9d7036c6c6669ca788804d0
I20260806 18:30:36.305532 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:36.301090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:40051 (local address 127.7.67.254:46123)
0806 18:30:36.301261 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:36.301266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:36.302387 (+ 1121us) server_negotiation.cc:407] Connection header received
0806 18:30:36.303808 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:36.303812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:36.303862 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:36.303938 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:36.305073 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:36.305077 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:36.305079 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:36.305094 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:36.305120 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:36.305129 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:36.305213 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:36.305335 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 18:30:36.305385 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:30:36.306476 21684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:36.301538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:46123 (local address 127.7.67.193:40051)
0806 18:30:36.302229 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:36.302273 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:36.303586 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:36.304125 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:36.304133 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:36.304226 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:36.304801 (+ 575us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:30:36.304802 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:36.304844 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:30:36.304917 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:36.305195 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:36.305205 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:36.305211 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:30:36.305377 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 18:30:36.305655 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":114,"threads_started":1}
I20260806 18:30:36.307405 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:46123
--builtin_ntp_servers=127.7.67.212:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 18:30:36.307796 21679 heartbeater.cc:347] Connected to a master server at 127.7.67.254:46123
I20260806 18:30:36.308045 21679 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:36.308467 21679 heartbeater.cc:511] Master 127.7.67.254:46123 requested a full tablet report, sending...
I20260806 18:30:36.310113 21390 ts_manager.cc:194] Registered new tserver with Master: 91bd15068b9247e48b2d776b06e09e7f (127.7.67.193:46285)
W20260806 18:30:36.422843 21685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:36.423156 21685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:36.423197 21685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:36.427248 21685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:36.427310 21685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:36.427385 21685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:30:36.432178 21685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:46123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:36.433264 21685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:36.434702 21685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:36.444478 21691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:36.444478 21693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:36.444492 21690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:36.444856 21685 server_base.cc:1034] running on GCE node
I20260806 18:30:36.445283 21685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:36.445950 21685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:36.447106 21685 hybrid_clock.cc:648] HybridClock initialized: now 1786041036447085 us; error 32 us; skew 500 ppm
I20260806 18:30:36.449021 21685 webserver.cc:460] Webserver started at http://127.7.67.194:45929/ using document root <none> and password file <none>
I20260806 18:30:36.449621 21685 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:36.449688 21685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:36.449906 21685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:36.452129 21685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "10c3f66558674cd5b45faa58f3a1d528"
format_stamp: "Formatted at 2026-08-06 18:30:36 on dist-test-slave-5bn2"
server_key: "00423c1415610d5c17e6d3db308f6caa"
server_key_iv: "7ccec1010380231c2c3b6313ec93a8c6"
server_key_version: "encryptionkey@0"
I20260806 18:30:36.452677 21685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "10c3f66558674cd5b45faa58f3a1d528"
format_stamp: "Formatted at 2026-08-06 18:30:36 on dist-test-slave-5bn2"
server_key: "00423c1415610d5c17e6d3db308f6caa"
server_key_iv: "7ccec1010380231c2c3b6313ec93a8c6"
server_key_version: "encryptionkey@0"
I20260806 18:30:36.456444 21685 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:30:36.459079 21702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:36.460338 21685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:30:36.460446 21685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "10c3f66558674cd5b45faa58f3a1d528"
format_stamp: "Formatted at 2026-08-06 18:30:36 on dist-test-slave-5bn2"
server_key: "00423c1415610d5c17e6d3db308f6caa"
server_key_iv: "7ccec1010380231c2c3b6313ec93a8c6"
server_key_version: "encryptionkey@0"
I20260806 18:30:36.460575 21685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:36.465231 21685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:36.465864 21685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:36.466024 21685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:36.466572 21685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:36.467545 21685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:36.467592 21685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:36.467635 21685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:36.467664 21685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:36.476099 21685 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:39269
I20260806 18:30:36.476154 21867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:39269 every 8 connection(s)
I20260806 18:30:36.477031 21685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:30:36.482947 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:36.478833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:46743 (local address 127.7.67.254:46123)
0806 18:30:36.478993 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:36.478997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:36.480061 (+ 1064us) server_negotiation.cc:407] Connection header received
0806 18:30:36.481387 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:36.481390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:36.481431 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:36.481517 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:36.482538 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:36.482542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:36.482545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:36.482558 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:36.482578 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:36.482587 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:36.482660 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:36.482750 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 18:30:36.482790 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 18:30:36.483812 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:36.479260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:46123 (local address 127.7.67.194:46743)
0806 18:30:36.479919 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:36.479960 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:36.481178 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:36.481669 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:36.481676 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:36.481778 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:36.482293 (+ 515us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:30:36.482294 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:36.482334 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:30:36.482402 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:36.482651 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:36.482655 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:36.482659 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:30:36.482783 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 18:30:36.483027 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":124,"threads_started":1}
I20260806 18:30:36.484931 21868 heartbeater.cc:347] Connected to a master server at 127.7.67.254:46123
I20260806 18:30:36.485076 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 21685
I20260806 18:30:36.485155 21868 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:36.485201 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:30:36.485481 7439 external_mini_cluster.cc:1428] Setting key 2a68163e3f4b27763dccf9f11aa54680
I20260806 18:30:36.485595 21868 heartbeater.cc:511] Master 127.7.67.254:46123 requested a full tablet report, sending...
I20260806 18:30:36.486677 21389 ts_manager.cc:194] Registered new tserver with Master: 10c3f66558674cd5b45faa58f3a1d528 (127.7.67.194:39269)
I20260806 18:30:36.487380 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:46123
--builtin_ntp_servers=127.7.67.212:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 18:30:36.603884 21872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:36.604197 21872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:36.604238 21872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:36.608381 21872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:36.608482 21872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:36.608543 21872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:30:36.613610 21872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:46123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:36.614811 21872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:36.616432 21872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:36.626533 21879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:36.626559 21878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:36.626554 21881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:36.626729 21872 server_base.cc:1034] running on GCE node
I20260806 18:30:36.627352 21872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:36.628028 21872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:36.629196 21872 hybrid_clock.cc:648] HybridClock initialized: now 1786041036629173 us; error 37 us; skew 500 ppm
I20260806 18:30:36.631194 21872 webserver.cc:460] Webserver started at http://127.7.67.195:43033/ using document root <none> and password file <none>
I20260806 18:30:36.631799 21872 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:36.631871 21872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:36.632094 21872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:36.634274 21872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "6267f5b5deb54b31b9123e3d98723fff"
format_stamp: "Formatted at 2026-08-06 18:30:36 on dist-test-slave-5bn2"
server_key: "1bd7657a8cd87c7a36d2d5920667a33c"
server_key_iv: "fea85cd01113ad3ec944442a3eee9bb1"
server_key_version: "encryptionkey@0"
I20260806 18:30:36.634871 21872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "6267f5b5deb54b31b9123e3d98723fff"
format_stamp: "Formatted at 2026-08-06 18:30:36 on dist-test-slave-5bn2"
server_key: "1bd7657a8cd87c7a36d2d5920667a33c"
server_key_iv: "fea85cd01113ad3ec944442a3eee9bb1"
server_key_version: "encryptionkey@0"
I20260806 18:30:36.638629 21872 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:30:36.641445 21887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:36.642736 21872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:30:36.642889 21872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "6267f5b5deb54b31b9123e3d98723fff"
format_stamp: "Formatted at 2026-08-06 18:30:36 on dist-test-slave-5bn2"
server_key: "1bd7657a8cd87c7a36d2d5920667a33c"
server_key_iv: "fea85cd01113ad3ec944442a3eee9bb1"
server_key_version: "encryptionkey@0"
I20260806 18:30:36.643005 21872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:36.647800 21872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:36.648443 21872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:36.648614 21872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:36.649194 21872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:36.650205 21872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:36.650254 21872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:36.650307 21872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:36.650339 21872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:36.659022 21872 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:36699
I20260806 18:30:36.659044 22053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:36699 every 8 connection(s)
I20260806 18:30:36.660010 21872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:30:36.664302 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 21872
I20260806 18:30:36.664431 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:30:36.664765 7439 external_mini_cluster.cc:1428] Setting key 31fd4f50a6f256501cf8ffb82c4d8916
I20260806 18:30:36.666329 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:36.661853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:45539 (local address 127.7.67.254:46123)
0806 18:30:36.661999 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:36.662003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:36.663192 (+ 1189us) server_negotiation.cc:407] Connection header received
0806 18:30:36.664634 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:36.664638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:36.664688 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:36.664767 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:36.665863 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:36.665866 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:36.665869 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:30:36.665882 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:36.665906 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:30:36.665916 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:36.665998 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:36.666149 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 18:30:36.666195 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:30:36.667266 22061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:36.662277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:46123 (local address 127.7.67.195:45539)
0806 18:30:36.663045 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:36.663089 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:36.664324 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:36.664958 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:36.664965 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:36.665057 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:36.665613 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:30:36.665614 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:36.665657 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:30:36.665728 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:36.666001 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:36.666013 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:36.666019 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:30:36.666189 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 18:30:36.666442 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":121,"threads_started":1}
I20260806 18:30:36.668253 22058 heartbeater.cc:347] Connected to a master server at 127.7.67.254:46123
I20260806 18:30:36.668452 22058 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:36.668829 22058 heartbeater.cc:511] Master 127.7.67.254:46123 requested a full tablet report, sending...
I20260806 18:30:36.669770 21389 ts_manager.cc:194] Registered new tserver with Master: 6267f5b5deb54b31b9123e3d98723fff (127.7.67.195:36699)
I20260806 18:30:36.678617 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 18:30:36.681494 21450 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:30:36.680553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44426 (local address 127.7.67.254:46123)
0806 18:30:36.680674 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:36.680678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:36.680866 (+ 188us) server_negotiation.cc:407] Connection header received
0806 18:30:36.680998 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:36.681001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:36.681040 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:36.681104 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:36.681350 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44426: BlockingRecv error: recv got EOF from 127.0.0.1:44426 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:30:36.681587 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 21488
I20260806 18:30:36.702471 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 21685
I20260806 18:30:36.711061 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 21872
I20260806 18:30:36.725335 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 21344
2026-08-06T18:30:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (862 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:30:36 dist-test-slave-5bn2 krb5kdc[22073](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:30:36 dist-test-slave-5bn2 krb5kdc[22073](info): set up 2 sockets
Aug 06 18:30:36 dist-test-slave-5bn2 krb5kdc[22073](info): commencing operation
krb5kdc: starting...
W20260806 18:30:37.850767 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:30:37 dist-test-slave-5bn2 krb5kdc[22073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:30:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:30:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:30:37.975507 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:37139
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:41729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:37139
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:38.088132 22088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:38.088421 22088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:38.088480 22088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:38.092541 22088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:30:38.092607 22088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:38.092633 22088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:38.092655 22088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:30:38.092684 22088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:30:38.098033 22088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:41729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:37139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:37139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:38.099181 22088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:38.100234 22088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:38.108656 22097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:38.108649 22098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:38.108649 22102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:38.108865 22088 server_base.cc:1034] running on GCE node
I20260806 18:30:38.109459 22088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:38.110598 22088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:38.111754 22088 hybrid_clock.cc:648] HybridClock initialized: now 1786041038111727 us; error 38 us; skew 500 ppm
Aug 06 18:30:38 dist-test-slave-5bn2 krb5kdc[22073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041038, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:38.114418 22088 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:30:38.115377 22088 webserver.cc:460] Webserver started at http://127.7.67.254:46651/ using document root <none> and password file <none>
I20260806 18:30:38.115935 22088 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:38.116006 22088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:38.116202 22088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:38.118255 22088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "8941a8d9e5814fd0bb18bac4d4e4a7e2"
format_stamp: "Formatted at 2026-08-06 18:30:38 on dist-test-slave-5bn2"
server_key: "3eaeeaba8d03481563a1f4b376afb335"
server_key_iv: "0d45b9b99fddb729f3d7431ed8b8b7f9"
server_key_version: "encryptionkey@0"
I20260806 18:30:38.118814 22088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "8941a8d9e5814fd0bb18bac4d4e4a7e2"
format_stamp: "Formatted at 2026-08-06 18:30:38 on dist-test-slave-5bn2"
server_key: "3eaeeaba8d03481563a1f4b376afb335"
server_key_iv: "0d45b9b99fddb729f3d7431ed8b8b7f9"
server_key_version: "encryptionkey@0"
I20260806 18:30:38.122479 22088 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:30:38.124872 22114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:38.126062 22088 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:30:38.126190 22088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "8941a8d9e5814fd0bb18bac4d4e4a7e2"
format_stamp: "Formatted at 2026-08-06 18:30:38 on dist-test-slave-5bn2"
server_key: "3eaeeaba8d03481563a1f4b376afb335"
server_key_iv: "0d45b9b99fddb729f3d7431ed8b8b7f9"
server_key_version: "encryptionkey@0"
I20260806 18:30:38.126284 22088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:38.137603 22088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:38.140489 22088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:38.140679 22088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:38.147656 22088 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:37139
I20260806 18:30:38.147909 22199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:37139 every 8 connection(s)
I20260806 18:30:38.148663 22088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:30:38.151515 22200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:38.152127 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 22088
I20260806 18:30:38.152227 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:30:38.152505 7439 external_mini_cluster.cc:1428] Setting key 1484c090a729623f498bde995c85991f
I20260806 18:30:38.157761 22200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2: Bootstrap starting.
I20260806 18:30:38.160547 22200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:38.161399 22200 log.cc:824] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:30:38 dist-test-slave-5bn2 krb5kdc[22073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:30:38.163512 22200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2: No bootstrap required, opened a new log
I20260806 18:30:38.166446 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:38.153904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50066 (local address 127.7.67.254:37139)
0806 18:30:38.154339 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:38.154355 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:38.154392 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:30:38.155097 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:38.155116 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:38.155450 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:38.155740 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:38.156641 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.157599 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.158716 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.159237 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.159357 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:38.162063 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:38.162069 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:38.162084 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:38.162117 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:38.163725 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:38.164149 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:38.164154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:38.164159 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:38.164274 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:38.164550 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:38.164553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:38.164554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:38.164946 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:38.165123 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:38.165426 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 18:30:38.165723 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":129,"threads_started":1}
I20260806 18:30:38.167157 22200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8941a8d9e5814fd0bb18bac4d4e4a7e2" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 37139 } }
I20260806 18:30:38.167379 22200 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:38.167439 22200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8941a8d9e5814fd0bb18bac4d4e4a7e2, State: Initialized, Role: FOLLOWER
I20260806 18:30:38.167877 22200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8941a8d9e5814fd0bb18bac4d4e4a7e2" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 37139 } }
I20260806 18:30:38.168020 22200 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:30:38.168082 22200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:30:38.168179 22200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:38.169195 22200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8941a8d9e5814fd0bb18bac4d4e4a7e2" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 37139 } }
I20260806 18:30:38.169519 22200 leader_election.cc:302] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8941a8d9e5814fd0bb18bac4d4e4a7e2; no voters:
I20260806 18:30:38.169809 22200 leader_election.cc:288] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:30:38.169948 22207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:38.170192 22207 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2 [term 1 LEADER]: Becoming Leader. State: Replica: 8941a8d9e5814fd0bb18bac4d4e4a7e2, State: Running, Role: LEADER
I20260806 18:30:38.170514 22207 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8941a8d9e5814fd0bb18bac4d4e4a7e2" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 37139 } }
I20260806 18:30:38.171039 22200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:30:38.172170 22208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8941a8d9e5814fd0bb18bac4d4e4a7e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8941a8d9e5814fd0bb18bac4d4e4a7e2" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 37139 } } }
I20260806 18:30:38.172302 22208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:38.172528 22209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8941a8d9e5814fd0bb18bac4d4e4a7e2. Latest consensus state: current_term: 1 leader_uuid: "8941a8d9e5814fd0bb18bac4d4e4a7e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8941a8d9e5814fd0bb18bac4d4e4a7e2" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 37139 } } }
I20260806 18:30:38.172680 22209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:30:38.172705 22217 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:30:38.175629 22217 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:30:38.180819 22217 catalog_manager.cc:1347] Generated new cluster ID: a00ac9360dd14289819cb4d7d2c52c9a
I20260806 18:30:38.180938 22217 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:30:38.191195 22217 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:30:38.192154 22217 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:30:38.199584 22217 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2: Generated new TSK 0
I20260806 18:30:38.200287 22217 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
I20260806 18:30:38.230523 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:37139
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:41729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:38.344767 22235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:38.345011 22235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:38.345054 22235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:38.349018 22235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:38.349078 22235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:38.349143 22235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:30:38.354321 22235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:41729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:37139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:38.355393 22235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:38.356429 22235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:38.365921 22241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:38.365916 22240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:38.366107 22235 server_base.cc:1034] running on GCE node
W20260806 18:30:38.366291 22243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:38.366753 22235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:38.367437 22235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:38.368602 22235 hybrid_clock.cc:648] HybridClock initialized: now 1786041038368584 us; error 30 us; skew 500 ppm
Aug 06 18:30:38 dist-test-slave-5bn2 krb5kdc[22073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041038, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:38.371506 22235 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:30:38.372426 22235 webserver.cc:460] Webserver started at http://127.7.67.193:45221/ using document root <none> and password file <none>
I20260806 18:30:38.372994 22235 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:38.373057 22235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:38.373234 22235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:38.375320 22235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "ae608f372b434251a86b53272100add4"
format_stamp: "Formatted at 2026-08-06 18:30:38 on dist-test-slave-5bn2"
server_key: "0e8bf78cb58983ec2c648430947ae8a5"
server_key_iv: "7160e55e99ec94a8c95114f0d33e1218"
server_key_version: "encryptionkey@0"
I20260806 18:30:38.375854 22235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "ae608f372b434251a86b53272100add4"
format_stamp: "Formatted at 2026-08-06 18:30:38 on dist-test-slave-5bn2"
server_key: "0e8bf78cb58983ec2c648430947ae8a5"
server_key_iv: "7160e55e99ec94a8c95114f0d33e1218"
server_key_version: "encryptionkey@0"
I20260806 18:30:38.379707 22235 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:30:38.382226 22253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:38.383574 22235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:30:38.383690 22235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "ae608f372b434251a86b53272100add4"
format_stamp: "Formatted at 2026-08-06 18:30:38 on dist-test-slave-5bn2"
server_key: "0e8bf78cb58983ec2c648430947ae8a5"
server_key_iv: "7160e55e99ec94a8c95114f0d33e1218"
server_key_version: "encryptionkey@0"
I20260806 18:30:38.383790 22235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:38.399108 22235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:38.403363 22235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:38.403586 22235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:38.404378 22235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:38.405527 22235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:38.405597 22235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:38.405666 22235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:38.405697 22235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:38.415417 22235 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:36323
I20260806 18:30:38.415444 22431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:36323 every 8 connection(s)
I20260806 18:30:38.416386 22235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:30:38.417546 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 22235
I20260806 18:30:38.417680 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:30:38.418012 7439 external_mini_cluster.cc:1428] Setting key 24a1dda69fa3a9c6064eae1abe50c28f
Aug 06 18:30:38 dist-test-slave-5bn2 krb5kdc[22073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041038, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.194@KRBTEST.COM" created.
I20260806 18:30:38.431761 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:38.418422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:40097 (local address 127.7.67.254:37139)
0806 18:30:38.418582 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:38.418587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:38.419631 (+ 1044us) server_negotiation.cc:407] Connection header received
0806 18:30:38.421027 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:38.421031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:38.421085 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:38.421156 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:38.422734 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.423355 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.424975 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.425208 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.425286 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:38.428120 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:38.428124 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:38.428126 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:38.428151 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:38.429550 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:38.430124 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:38.430128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:38.430130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:38.430210 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:38.430555 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:38.430567 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:38.430568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:38.430757 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:38.430870 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:38.431437 (+ 567us) server_negotiation.cc:299] Negotiation successful
0806 18:30:38.431586 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:30:38.432818 22435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:38.418910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:37139 (local address 127.7.67.193:40097)
0806 18:30:38.419476 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:38.419535 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:38.420825 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:38.421339 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:38.421349 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:38.421865 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:38.422561 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:38.422582 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.423490 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.423493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:38.424858 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:38.424870 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.425374 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.425379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:38.425597 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:38.426224 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:38.426246 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:38.427938 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:38.429670 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:38.429676 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:38.429693 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:38.430021 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:38.430316 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:38.430320 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:38.430322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:38.430465 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:38.430891 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:38.430899 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:38.431197 (+ 298us) client_negotiation.cc:769] Sending connection context
0806 18:30:38.431377 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 18:30:38.431729 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":132,"threads_started":1}
I20260806 18:30:38.434165 22432 heartbeater.cc:347] Connected to a master server at 127.7.67.254:37139
I20260806 18:30:38.434480 22432 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:38.435132 22432 heartbeater.cc:511] Master 127.7.67.254:37139 requested a full tablet report, sending...
I20260806 18:30:38.436815 22145 ts_manager.cc:194] Registered new tserver with Master: ae608f372b434251a86b53272100add4 (127.7.67.193:36323)
I20260806 18:30:38.437963 22145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:40097
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
I20260806 18:30:38.448261 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:37139
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:41729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:38.561913 22439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:38.562181 22439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:38.562229 22439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:38.566181 22439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:38.566248 22439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:38.566311 22439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:30:38.571619 22439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:41729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:37139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:38.572718 22439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:38.573773 22439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:38.583178 22452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:38.583199 22449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:38.583185 22450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:38.583552 22439 server_base.cc:1034] running on GCE node
I20260806 18:30:38.583956 22439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:38.584597 22439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:38.585772 22439 hybrid_clock.cc:648] HybridClock initialized: now 1786041038585747 us; error 36 us; skew 500 ppm
Aug 06 18:30:38 dist-test-slave-5bn2 krb5kdc[22073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041038, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:38.588598 22439 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:30:38.589484 22439 webserver.cc:460] Webserver started at http://127.7.67.194:43391/ using document root <none> and password file <none>
I20260806 18:30:38.590013 22439 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:38.590083 22439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:38.590260 22439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:38.592301 22439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "faa3801b8b04485a8a2a5032626e89e7"
format_stamp: "Formatted at 2026-08-06 18:30:38 on dist-test-slave-5bn2"
server_key: "32fef2d2382dfe22d5505aaf2226442c"
server_key_iv: "a9f48a47a9d1d0a5680febaead597caa"
server_key_version: "encryptionkey@0"
I20260806 18:30:38.592824 22439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "faa3801b8b04485a8a2a5032626e89e7"
format_stamp: "Formatted at 2026-08-06 18:30:38 on dist-test-slave-5bn2"
server_key: "32fef2d2382dfe22d5505aaf2226442c"
server_key_iv: "a9f48a47a9d1d0a5680febaead597caa"
server_key_version: "encryptionkey@0"
I20260806 18:30:38.596565 22439 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:30:38.598901 22460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:38.600096 22439 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:30:38.600201 22439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "faa3801b8b04485a8a2a5032626e89e7"
format_stamp: "Formatted at 2026-08-06 18:30:38 on dist-test-slave-5bn2"
server_key: "32fef2d2382dfe22d5505aaf2226442c"
server_key_iv: "a9f48a47a9d1d0a5680febaead597caa"
server_key_version: "encryptionkey@0"
I20260806 18:30:38.600286 22439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:38.610702 22439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:38.613487 22439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:38.613638 22439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:38.614163 22439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:38.615003 22439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:38.615054 22439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:38.615108 22439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:38.615134 22439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:38.623346 22439 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:42823
I20260806 18:30:38.623386 22638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:42823 every 8 connection(s)
I20260806 18:30:38.624228 22439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:30:38.625221 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 22439
I20260806 18:30:38.625344 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:30:38.625623 7439 external_mini_cluster.cc:1428] Setting key 18d4d8f81207d408ff7a7085080c6e06
Aug 06 18:30:38 dist-test-slave-5bn2 krb5kdc[22073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041038, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.195@KRBTEST.COM; defaulting to no policy
I20260806 18:30:38.638389 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:38.626057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:51289 (local address 127.7.67.254:37139)
0806 18:30:38.626199 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:38.626204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:38.627101 (+ 897us) server_negotiation.cc:407] Connection header received
0806 18:30:38.628500 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:38.628503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:38.628554 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:38.628638 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:38.630108 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.630706 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.631860 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.632094 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.632184 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:38.634829 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:38.634833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:38.634836 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:38.634887 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:38.636368 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:38.636914 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:38.636920 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:38.636921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:38.637016 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:38.637302 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:38.637304 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:38.637305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:38.637546 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:38.637637 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:38.638116 (+ 479us) server_negotiation.cc:299] Negotiation successful
0806 18:30:38.638232 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.195@KRBTEST.COM" created.
I20260806 18:30:38.639338 22641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:38.626420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:37139 (local address 127.7.67.194:51289)
0806 18:30:38.626949 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:38.627015 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:38.628290 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:38.628797 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:38.628804 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:38.629265 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:38.629930 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:38.629950 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.630821 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.630824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:38.631736 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:38.631746 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.632240 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.632244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:38.632400 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:38.632979 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:38.632995 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:38.634644 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:38.636481 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:38.636486 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:38.636507 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:38.636786 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:38.637105 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:38.637107 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:38.637109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:38.637235 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:38.637639 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:38.637645 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:38.637891 (+ 246us) client_negotiation.cc:769] Sending connection context
0806 18:30:38.638053 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 18:30:38.638339 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":131,"threads_started":1}
I20260806 18:30:38.640563 22639 heartbeater.cc:347] Connected to a master server at 127.7.67.254:37139
I20260806 18:30:38.640863 22639 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:38.641441 22639 heartbeater.cc:511] Master 127.7.67.254:37139 requested a full tablet report, sending...
I20260806 18:30:38.642580 22145 ts_manager.cc:194] Registered new tserver with Master: faa3801b8b04485a8a2a5032626e89e7 (127.7.67.194:42823)
I20260806 18:30:38.643373 22145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:51289
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
I20260806 18:30:38.655566 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:37139
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:41729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:30:38.768754 22645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:30:38.769042 22645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:30:38.769085 22645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:30:38.773180 22645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:30:38.773242 22645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:30:38.773320 22645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:30:38.778581 22645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:41729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:37139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:30:38.779695 22645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:30:38.780742 22645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:30:38.790740 22656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:38.790783 22645 server_base.cc:1034] running on GCE node
W20260806 18:30:38.790740 22655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:30:38.790740 22661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:30:38.791416 22645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:30:38.792130 22645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:30:38.793295 22645 hybrid_clock.cc:648] HybridClock initialized: now 1786041038793273 us; error 34 us; skew 500 ppm
Aug 06 18:30:38 dist-test-slave-5bn2 krb5kdc[22073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041038, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:30:38.796072 22645 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:30:38.796991 22645 webserver.cc:460] Webserver started at http://127.7.67.195:37521/ using document root <none> and password file <none>
I20260806 18:30:38.797561 22645 fs_manager.cc:359] Metadata directory not provided
I20260806 18:30:38.797629 22645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:30:38.797833 22645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:30:38.799968 22645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "2ffc9fc7f46b45dc97a594013e3a8dcb"
format_stamp: "Formatted at 2026-08-06 18:30:38 on dist-test-slave-5bn2"
server_key: "e9371613e3854b0174ed22c92f2da89e"
server_key_iv: "26ea9066f983161e781db09bdd9d59b1"
server_key_version: "encryptionkey@0"
I20260806 18:30:38.800554 22645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "2ffc9fc7f46b45dc97a594013e3a8dcb"
format_stamp: "Formatted at 2026-08-06 18:30:38 on dist-test-slave-5bn2"
server_key: "e9371613e3854b0174ed22c92f2da89e"
server_key_iv: "26ea9066f983161e781db09bdd9d59b1"
server_key_version: "encryptionkey@0"
I20260806 18:30:38.804402 22645 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:30:38.806782 22671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:38.808051 22645 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:30:38.808173 22645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "2ffc9fc7f46b45dc97a594013e3a8dcb"
format_stamp: "Formatted at 2026-08-06 18:30:38 on dist-test-slave-5bn2"
server_key: "e9371613e3854b0174ed22c92f2da89e"
server_key_iv: "26ea9066f983161e781db09bdd9d59b1"
server_key_version: "encryptionkey@0"
I20260806 18:30:38.808276 22645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:30:38.820387 22645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:30:38.823586 22645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:30:38.823765 22645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:30:38.824373 22645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:30:38.825202 22645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:30:38.825256 22645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:38.825300 22645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:30:38.825327 22645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:30:38.834048 22645 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:35213
I20260806 18:30:38.834069 22837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:35213 every 8 connection(s)
I20260806 18:30:38.834977 22645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:30:38.842868 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 22645
I20260806 18:30:38.842985 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:30:38.843269 7439 external_mini_cluster.cc:1428] Setting key c31d3c39c9af612b5ec708e3050782b4
Aug 06 18:30:38 dist-test-slave-5bn2 krb5kdc[22073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041038, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:30:38.848062 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:38.836707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:45349 (local address 127.7.67.254:37139)
0806 18:30:38.836836 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:38.836840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:38.837679 (+ 839us) server_negotiation.cc:407] Connection header received
0806 18:30:38.839074 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:38.839078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:38.839138 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:38.839212 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:38.840420 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.840954 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.842089 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.842299 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.842385 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:38.845082 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:38.845085 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:38.845087 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:38.845112 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:38.846322 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:38.846780 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:38.846782 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:38.846783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:38.846893 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:38.847166 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:38.847168 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:38.847169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:38.847339 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:38.847408 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:38.847838 (+ 430us) server_negotiation.cc:299] Negotiation successful
0806 18:30:38.847933 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:30:38.848941 22840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:38.837069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:37139 (local address 127.7.67.195:45349)
0806 18:30:38.837524 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:38.837569 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:38.838824 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:38.839399 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:38.839406 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:38.839773 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:30:38.840265 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:38.840279 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.841077 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.841080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:38.841963 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:38.841973 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.842443 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.842447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:38.842612 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:38.843222 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:30:38.843238 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:30:38.844938 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:30:38.846423 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:38.846428 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:38.846440 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:38.846706 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:38.846963 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:30:38.846965 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:30:38.846966 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:30:38.847100 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:30:38.847407 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:30:38.847412 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:30:38.847655 (+ 243us) client_negotiation.cc:769] Sending connection context
0806 18:30:38.847808 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 18:30:38.848066 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":112,"threads_started":1}
I20260806 18:30:38.849972 22838 heartbeater.cc:347] Connected to a master server at 127.7.67.254:37139
I20260806 18:30:38.850196 22838 heartbeater.cc:464] Registering TS with master...
I20260806 18:30:38.850664 22838 heartbeater.cc:511] Master 127.7.67.254:37139 requested a full tablet report, sending...
I20260806 18:30:38.851670 22145 ts_manager.cc:194] Registered new tserver with Master: 2ffc9fc7f46b45dc97a594013e3a8dcb (127.7.67.195:35213)
I20260806 18:30:38.852206 22145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:45349
I20260806 18:30:38.857069 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:30:38.865852 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:38.858956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50076 (local address 127.7.67.254:37139)
0806 18:30:38.859088 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:38.859093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:38.859279 (+ 186us) server_negotiation.cc:407] Connection header received
0806 18:30:38.859408 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:38.859411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:38.859455 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:38.859514 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:30:38.860189 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.860702 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.861523 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.861748 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.861835 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:38.863130 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:30:38.863134 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:30:38.863137 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:30:38.863163 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:30:38.864236 (+ 1073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:38.864640 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:38.864644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:38.864646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:38.864730 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:30:38.864976 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:30:38.864979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:30:38.864981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:30:38.865220 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:30:38.865343 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:38.865579 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 18:30:38.865693 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:30:38.868660 22145 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:30:38.871855 22145 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:30:38.887434 22853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:38.881056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49514 (local address 127.7.67.195:35213)
0806 18:30:38.881408 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:38.881420 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:38.882345 (+ 925us) server_negotiation.cc:407] Connection header received
0806 18:30:38.882434 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:38.882439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:38.882652 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:38.882797 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:38.883229 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.884233 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.886112 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.886667 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.886767 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:38.886971 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:38.887122 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 18:30:38.887218 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":116,"threads_started":1}
I20260806 18:30:38.887426 22854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:38.880850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:35213 (local address 127.0.0.1:49514)
0806 18:30:38.882009 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:38.882061 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:38.882205 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:38.882932 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:38.882939 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:38.882961 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:38.883109 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:38.883116 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.884338 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.884341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:38.885964 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:38.885976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.886783 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.886787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:38.886922 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:38.886963 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 18:30:38.887084 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 18:30:38.887216 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":957,"mutex_wait_us":128,"thread_start_us":98,"threads_started":1}
I20260806 18:30:38.887902 22860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:38.881948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:42823 (local address 127.0.0.1:53592)
0806 18:30:38.882669 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:38.882679 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:38.882767 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:38.883377 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:38.883380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:38.883399 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:38.883485 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:38.883491 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.885232 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.885235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:38.886010 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:38.886019 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.887525 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.887528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:38.887617 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:38.887642 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:30:38.887703 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 18:30:38.887767 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"mutex_wait_us":4,"thread_start_us":100,"threads_started":1}
I20260806 18:30:38.888343 22862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:38.882135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53592 (local address 127.7.67.194:42823)
0806 18:30:38.882561 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:38.882569 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:38.882771 (+ 202us) server_negotiation.cc:407] Connection header received
0806 18:30:38.882883 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:38.882889 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:38.883071 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:38.883220 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:38.883623 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.885105 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.886145 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.887225 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.887320 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:38.887463 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:38.888094 (+ 631us) server_negotiation.cc:299] Negotiation successful
0806 18:30:38.888197 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":153,"threads_started":1}
I20260806 18:30:38.888679 22852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:38.881607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:36323 (local address 127.0.0.1:44634)
0806 18:30:38.882236 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:38.882250 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:38.882352 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:38.883118 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:38.883120 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:38.883135 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:38.883228 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:38.883234 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.884297 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.884301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:38.885964 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:38.885976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.888244 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.888248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:38.888426 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:38.888463 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 18:30:38.888529 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 18:30:38.888587 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":170,"thread_start_us":108,"threads_started":1}
I20260806 18:30:38.889163 22859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:38.881821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44634 (local address 127.7.67.193:36323)
0806 18:30:38.882199 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:38.882214 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:38.882575 (+ 361us) server_negotiation.cc:407] Connection header received
0806 18:30:38.882658 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:38.882663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:38.882826 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:38.882986 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:38.883352 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.884177 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.886096 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:38.886951 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:38.888814 (+ 1863us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:38.888941 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:38.888984 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 18:30:38.889054 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":95,"threads_started":1}
I20260806 18:30:38.890533 22741 tablet_service.cc:1467] Processing CreateTablet for tablet b578fb53ecd645c2863e8230cd04bf25 (DEFAULT_TABLE table=test-table [id=f04a0a725ec24cea9cdfa30cbccca651]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:38.891848 22741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b578fb53ecd645c2863e8230cd04bf25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:38.892768 22536 tablet_service.cc:1467] Processing CreateTablet for tablet b578fb53ecd645c2863e8230cd04bf25 (DEFAULT_TABLE table=test-table [id=f04a0a725ec24cea9cdfa30cbccca651]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:38.893682 22536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b578fb53ecd645c2863e8230cd04bf25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:38.897956 22327 tablet_service.cc:1467] Processing CreateTablet for tablet b578fb53ecd645c2863e8230cd04bf25 (DEFAULT_TABLE table=test-table [id=f04a0a725ec24cea9cdfa30cbccca651]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:30:38.898521 22865 tablet_bootstrap.cc:492] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb: Bootstrap starting.
I20260806 18:30:38.899148 22327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b578fb53ecd645c2863e8230cd04bf25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:30:38.900494 22867 tablet_bootstrap.cc:492] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7: Bootstrap starting.
I20260806 18:30:38.901257 22865 tablet_bootstrap.cc:654] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:38.902616 22865 log.cc:824] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:38.903450 22867 tablet_bootstrap.cc:654] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:38.904343 22867 log.cc:824] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:38.904943 22865 tablet_bootstrap.cc:492] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb: No bootstrap required, opened a new log
I20260806 18:30:38.905148 22865 ts_tablet_manager.cc:1397] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 18:30:38.906416 22872 tablet_bootstrap.cc:492] T b578fb53ecd645c2863e8230cd04bf25 P ae608f372b434251a86b53272100add4: Bootstrap starting.
I20260806 18:30:38.906458 22867 tablet_bootstrap.cc:492] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7: No bootstrap required, opened a new log
I20260806 18:30:38.906652 22867 ts_tablet_manager.cc:1397] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 18:30:38.908439 22872 tablet_bootstrap.cc:654] T b578fb53ecd645c2863e8230cd04bf25 P ae608f372b434251a86b53272100add4: Neither blocks nor log segments found. Creating new log.
I20260806 18:30:38.909142 22865 raft_consensus.cc:348] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffc9fc7f46b45dc97a594013e3a8dcb" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35213 } } peers { permanent_uuid: "ae608f372b434251a86b53272100add4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36323 } } peers { permanent_uuid: "faa3801b8b04485a8a2a5032626e89e7" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42823 } }
I20260806 18:30:38.909432 22865 raft_consensus.cc:374] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:38.909492 22865 raft_consensus.cc:729] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ffc9fc7f46b45dc97a594013e3a8dcb, State: Initialized, Role: FOLLOWER
I20260806 18:30:38.909746 22872 log.cc:824] T b578fb53ecd645c2863e8230cd04bf25 P ae608f372b434251a86b53272100add4: Log is configured to *not* fsync() on all Append() calls
I20260806 18:30:38.910035 22865 consensus_queue.cc:260] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffc9fc7f46b45dc97a594013e3a8dcb" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35213 } } peers { permanent_uuid: "ae608f372b434251a86b53272100add4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36323 } } peers { permanent_uuid: "faa3801b8b04485a8a2a5032626e89e7" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42823 } }
I20260806 18:30:38.910517 22867 raft_consensus.cc:348] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffc9fc7f46b45dc97a594013e3a8dcb" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35213 } } peers { permanent_uuid: "ae608f372b434251a86b53272100add4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36323 } } peers { permanent_uuid: "faa3801b8b04485a8a2a5032626e89e7" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42823 } }
I20260806 18:30:38.910770 22867 raft_consensus.cc:374] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:38.910827 22867 raft_consensus.cc:729] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faa3801b8b04485a8a2a5032626e89e7, State: Initialized, Role: FOLLOWER
I20260806 18:30:38.911168 22865 ts_tablet_manager.cc:1428] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 18:30:38.911316 22867 consensus_queue.cc:260] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffc9fc7f46b45dc97a594013e3a8dcb" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35213 } } peers { permanent_uuid: "ae608f372b434251a86b53272100add4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36323 } } peers { permanent_uuid: "faa3801b8b04485a8a2a5032626e89e7" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42823 } }
I20260806 18:30:38.911779 22872 tablet_bootstrap.cc:492] T b578fb53ecd645c2863e8230cd04bf25 P ae608f372b434251a86b53272100add4: No bootstrap required, opened a new log
I20260806 18:30:38.911974 22872 ts_tablet_manager.cc:1397] T b578fb53ecd645c2863e8230cd04bf25 P ae608f372b434251a86b53272100add4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:30:38.911979 22639 heartbeater.cc:503] Master 127.7.67.254:37139 was elected leader, sending a full tablet report...
I20260806 18:30:38.913278 22867 ts_tablet_manager.cc:1428] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 18:30:38.915091 22838 heartbeater.cc:503] Master 127.7.67.254:37139 was elected leader, sending a full tablet report...
I20260806 18:30:38.915964 22872 raft_consensus.cc:348] T b578fb53ecd645c2863e8230cd04bf25 P ae608f372b434251a86b53272100add4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffc9fc7f46b45dc97a594013e3a8dcb" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35213 } } peers { permanent_uuid: "ae608f372b434251a86b53272100add4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36323 } } peers { permanent_uuid: "faa3801b8b04485a8a2a5032626e89e7" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42823 } }
I20260806 18:30:38.916225 22872 raft_consensus.cc:374] T b578fb53ecd645c2863e8230cd04bf25 P ae608f372b434251a86b53272100add4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:30:38.916277 22872 raft_consensus.cc:729] T b578fb53ecd645c2863e8230cd04bf25 P ae608f372b434251a86b53272100add4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae608f372b434251a86b53272100add4, State: Initialized, Role: FOLLOWER
I20260806 18:30:38.916739 22872 consensus_queue.cc:260] T b578fb53ecd645c2863e8230cd04bf25 P ae608f372b434251a86b53272100add4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffc9fc7f46b45dc97a594013e3a8dcb" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35213 } } peers { permanent_uuid: "ae608f372b434251a86b53272100add4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36323 } } peers { permanent_uuid: "faa3801b8b04485a8a2a5032626e89e7" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42823 } }
I20260806 18:30:38.917697 22872 ts_tablet_manager.cc:1428] T b578fb53ecd645c2863e8230cd04bf25 P ae608f372b434251a86b53272100add4: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:30:38.918103 22432 heartbeater.cc:503] Master 127.7.67.254:37139 was elected leader, sending a full tablet report...
I20260806 18:30:39.041982 22874 raft_consensus.cc:482] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:30:39.042207 22874 raft_consensus.cc:504] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffc9fc7f46b45dc97a594013e3a8dcb" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35213 } } peers { permanent_uuid: "ae608f372b434251a86b53272100add4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36323 } } peers { permanent_uuid: "faa3801b8b04485a8a2a5032626e89e7" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42823 } }
I20260806 18:30:39.043392 22874 leader_election.cc:288] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae608f372b434251a86b53272100add4 (127.7.67.193:36323), faa3801b8b04485a8a2a5032626e89e7 (127.7.67.194:42823)
I20260806 18:30:39.047042 22840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:39.043690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:36323 (local address 127.7.67.195:48099)
0806 18:30:39.043839 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:39.043857 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:39.043966 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:39.044312 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:39.044315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:39.044334 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:39.044434 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:39.044439 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:39.045151 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:39.045155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:39.045957 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:39.045966 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:39.046593 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:39.046596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:39.046703 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:39.046726 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:30:39.046791 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 18:30:39.046891 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":22}
I20260806 18:30:39.047041 22859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:39.043773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:48099 (local address 127.7.67.193:36323)
0806 18:30:39.043926 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:39.043931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:39.043949 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:30:39.044072 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:39.044076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:39.044130 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:39.044211 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:39.044553 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:39.045042 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:39.046091 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:39.046473 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:39.046588 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:39.046644 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:39.046819 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 18:30:39.046920 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:30:39.047391 22862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:39.044155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:57387 (local address 127.7.67.194:42823)
0806 18:30:39.044285 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:39.044289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:39.044443 (+ 154us) server_negotiation.cc:407] Connection header received
0806 18:30:39.044554 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:39.044558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:39.044601 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:39.044664 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:30:39.044917 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:39.045397 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:39.046416 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:39.046804 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:39.046945 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:39.047006 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:39.047182 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 18:30:39.047272 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:30:39.047757 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:39.043968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:42823 (local address 127.7.67.195:57387)
0806 18:30:39.044359 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:39.044369 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 18:30:39.044460 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:30:39.044684 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:30:39.044687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:30:39.044701 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:30:39.044811 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:39.044816 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:39.045508 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:39.045511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:39.046291 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:30:39.046301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:30:39.046942 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:30:39.046946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:30:39.047043 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:39.047065 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:30:39.047545 (+ 480us) client_negotiation.cc:240] Negotiation successful
0806 18:30:39.047618 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":3,"thread_start_us":145,"threads_started":1}
I20260806 18:30:39.047814 22361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b578fb53ecd645c2863e8230cd04bf25" candidate_uuid: "2ffc9fc7f46b45dc97a594013e3a8dcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae608f372b434251a86b53272100add4" is_pre_election: true
I20260806 18:30:39.048195 22361 raft_consensus.cc:2454] T b578fb53ecd645c2863e8230cd04bf25 P ae608f372b434251a86b53272100add4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ffc9fc7f46b45dc97a594013e3a8dcb in term 0.
I20260806 18:30:39.048378 22568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b578fb53ecd645c2863e8230cd04bf25" candidate_uuid: "2ffc9fc7f46b45dc97a594013e3a8dcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faa3801b8b04485a8a2a5032626e89e7" is_pre_election: true
I20260806 18:30:39.048687 22568 raft_consensus.cc:2454] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ffc9fc7f46b45dc97a594013e3a8dcb in term 0.
I20260806 18:30:39.048691 22674 leader_election.cc:302] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ffc9fc7f46b45dc97a594013e3a8dcb, ae608f372b434251a86b53272100add4; no voters:
I20260806 18:30:39.048982 22874 raft_consensus.cc:2781] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:30:39.049062 22874 raft_consensus.cc:482] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:30:39.049101 22874 raft_consensus.cc:3037] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:39.050164 22874 raft_consensus.cc:504] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffc9fc7f46b45dc97a594013e3a8dcb" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35213 } } peers { permanent_uuid: "ae608f372b434251a86b53272100add4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36323 } } peers { permanent_uuid: "faa3801b8b04485a8a2a5032626e89e7" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42823 } }
I20260806 18:30:39.050513 22874 leader_election.cc:288] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [CANDIDATE]: Term 1 election: Requested vote from peers ae608f372b434251a86b53272100add4 (127.7.67.193:36323), faa3801b8b04485a8a2a5032626e89e7 (127.7.67.194:42823)
I20260806 18:30:39.050870 22361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b578fb53ecd645c2863e8230cd04bf25" candidate_uuid: "2ffc9fc7f46b45dc97a594013e3a8dcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae608f372b434251a86b53272100add4"
I20260806 18:30:39.050951 22568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b578fb53ecd645c2863e8230cd04bf25" candidate_uuid: "2ffc9fc7f46b45dc97a594013e3a8dcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faa3801b8b04485a8a2a5032626e89e7"
I20260806 18:30:39.051002 22361 raft_consensus.cc:3037] T b578fb53ecd645c2863e8230cd04bf25 P ae608f372b434251a86b53272100add4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:39.051070 22568 raft_consensus.cc:3037] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:30:39.052069 22361 raft_consensus.cc:2454] T b578fb53ecd645c2863e8230cd04bf25 P ae608f372b434251a86b53272100add4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ffc9fc7f46b45dc97a594013e3a8dcb in term 1.
I20260806 18:30:39.052114 22568 raft_consensus.cc:2454] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ffc9fc7f46b45dc97a594013e3a8dcb in term 1.
I20260806 18:30:39.052320 22674 leader_election.cc:302] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ffc9fc7f46b45dc97a594013e3a8dcb, ae608f372b434251a86b53272100add4; no voters:
I20260806 18:30:39.052484 22874 raft_consensus.cc:2781] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:30:39.052718 22874 raft_consensus.cc:686] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [term 1 LEADER]: Becoming Leader. State: Replica: 2ffc9fc7f46b45dc97a594013e3a8dcb, State: Running, Role: LEADER
I20260806 18:30:39.053079 22874 consensus_queue.cc:237] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffc9fc7f46b45dc97a594013e3a8dcb" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35213 } } peers { permanent_uuid: "ae608f372b434251a86b53272100add4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36323 } } peers { permanent_uuid: "faa3801b8b04485a8a2a5032626e89e7" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42823 } }
I20260806 18:30:39.056120 22145 catalog_manager.cc:5495] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ffc9fc7f46b45dc97a594013e3a8dcb (127.7.67.195). New cstate: current_term: 1 leader_uuid: "2ffc9fc7f46b45dc97a594013e3a8dcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ffc9fc7f46b45dc97a594013e3a8dcb" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae608f372b434251a86b53272100add4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faa3801b8b04485a8a2a5032626e89e7" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42823 } health_report { overall_health: UNKNOWN } } }
I20260806 18:30:39.075070 22853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:39.071861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49518 (local address 127.7.67.195:35213)
0806 18:30:39.071987 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:30:39.071991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:30:39.072004 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:30:39.072081 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:30:39.072083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:30:39.072139 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:30:39.072223 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:30:39.072400 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:39.072913 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:39.073730 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:30:39.073939 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:30:39.074013 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:30:39.074264 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:30:39.074705 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:30:39.074793 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:30:39.074866 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 18:30:39.074931 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:30:39.081065 22568 raft_consensus.cc:1261] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7 [term 1 FOLLOWER]: Refusing update from remote peer 2ffc9fc7f46b45dc97a594013e3a8dcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:30:39.081065 22361 raft_consensus.cc:1261] T b578fb53ecd645c2863e8230cd04bf25 P ae608f372b434251a86b53272100add4 [term 1 FOLLOWER]: Refusing update from remote peer 2ffc9fc7f46b45dc97a594013e3a8dcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:30:39.081779 22874 consensus_queue.cc:1035] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "faa3801b8b04485a8a2a5032626e89e7" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:30:39.081990 22878 consensus_queue.cc:1035] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae608f372b434251a86b53272100add4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:30:39.086004 22839 tablet.cc:2369] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:30:39.091627 22883 mvcc.cc:205] Tried to move back new op lower bound from 7315624096068608000 to 7315624095963971584. Current Snapshot: MvccSnapshot[applied={T|T < 7315624096068608000}]
I20260806 18:30:39.097726 22145 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:30:39.097931 22145 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:30:39.100446 22145 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8941a8d9e5814fd0bb18bac4d4e4a7e2: Sending DeleteTablet for 3 replicas of tablet b578fb53ecd645c2863e8230cd04bf25
I20260806 18:30:39.101361 22741 tablet_service.cc:1514] Processing DeleteTablet for tablet b578fb53ecd645c2863e8230cd04bf25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:39 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:49514
I20260806 18:30:39.101542 22327 tablet_service.cc:1514] Processing DeleteTablet for tablet b578fb53ecd645c2863e8230cd04bf25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:39 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:44634
I20260806 18:30:39.101538 22536 tablet_service.cc:1514] Processing DeleteTablet for tablet b578fb53ecd645c2863e8230cd04bf25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:30:39 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:53592
I20260806 18:30:39.102003 22894 tablet_replica.cc:330] T b578fb53ecd645c2863e8230cd04bf25 P ae608f372b434251a86b53272100add4: stopping tablet replica
I20260806 18:30:39.101998 22893 tablet_replica.cc:330] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7: stopping tablet replica
I20260806 18:30:39.102325 22893 raft_consensus.cc:2229] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:30:39.102327 22894 raft_consensus.cc:2229] T b578fb53ecd645c2863e8230cd04bf25 P ae608f372b434251a86b53272100add4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:30:39.102653 22894 raft_consensus.cc:2258] T b578fb53ecd645c2863e8230cd04bf25 P ae608f372b434251a86b53272100add4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:39.102658 22893 raft_consensus.cc:2258] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:30:39.103003 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 22235
I20260806 18:30:39.104873 22893 ts_tablet_manager.cc:1905] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:39.107780 22893 ts_tablet_manager.cc:1918] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:30:39.107889 22893 log.cc:1196] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/b578fb53ecd645c2863e8230cd04bf25
I20260806 18:30:39.108232 22893 ts_tablet_manager.cc:1939] T b578fb53ecd645c2863e8230cd04bf25 P faa3801b8b04485a8a2a5032626e89e7: Deleting consensus metadata
I20260806 18:30:39.110191 22119 catalog_manager.cc:4841] TS faa3801b8b04485a8a2a5032626e89e7 (127.7.67.194:42823): tablet b578fb53ecd645c2863e8230cd04bf25 (table test-table [id=f04a0a725ec24cea9cdfa30cbccca651]) successfully deleted
I20260806 18:30:39.111285 22892 tablet_replica.cc:330] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb: stopping tablet replica
I20260806 18:30:39.111699 22892 raft_consensus.cc:2229] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:30:39.112097 22892 raft_consensus.cc:2258] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 18:30:39.117175 22119 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.193:36323 (error 108)
I20260806 18:30:39.117226 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 22439
I20260806 18:30:39.118470 22892 ts_tablet_manager.cc:1905] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:30:39.121301 22892 ts_tablet_manager.cc:1918] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:30:39.121400 22892 log.cc:1196] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/b578fb53ecd645c2863e8230cd04bf25
I20260806 18:30:39.121734 22892 ts_tablet_manager.cc:1939] T b578fb53ecd645c2863e8230cd04bf25 P 2ffc9fc7f46b45dc97a594013e3a8dcb: Deleting consensus metadata
I20260806 18:30:39.123241 22119 catalog_manager.cc:4841] TS 2ffc9fc7f46b45dc97a594013e3a8dcb (127.7.67.195:35213): tablet b578fb53ecd645c2863e8230cd04bf25 (table test-table [id=f04a0a725ec24cea9cdfa30cbccca651]) successfully deleted
I20260806 18:30:39.123416 22852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:30:39.123073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:36323 (local address 127.0.0.1:44636)
0806 18:30:39.123229 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 18:30:39.123319 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:36323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 18:30:39.123618 22119 catalog_manager.cc:4584] TS ae608f372b434251a86b53272100add4 (127.7.67.193:36323): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b578fb53ecd645c2863e8230cd04bf25: Network error: Client connection negotiation failed: client connection to 127.7.67.193:36323: connect: Connection refused (error 111)
I20260806 18:30:39.130586 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 22645
I20260806 18:30:39.145283 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 22088
2026-08-06T18:30:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2421 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:30:39 dist-test-slave-5bn2 krb5kdc[22901](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:30:39 dist-test-slave-5bn2 krb5kdc[22901](info): set up 2 sockets
Aug 06 18:30:39 dist-test-slave-5bn2 krb5kdc[22901](info): commencing operation
krb5kdc: starting...
W20260806 18:30:40.267554 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:30:40 dist-test-slave-5bn2 krb5kdc[22901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:30:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:30:40Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.67.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.67.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.67.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/rangeradmin_127.7.67.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.67.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/rangeradmin_127.7.67.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.67.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.67.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.67.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/rangerlookup_127.7.67.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.67.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/rangerlookup_127.7.67.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.67.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.67.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.67.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/HTTP_127.7.67.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.67.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/HTTP_127.7.67.212@KRBTEST.COM.keytab.
I20260806 18:30:40.441298 7439 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskM6OdK6/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/postgres -l logfile start
2026-08-06 18:30:42.888 UTC [22942] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 18:30:42.888 UTC [22942] LOG: listening on IPv4 address "127.7.67.212", port 41931
2026-08-06 18:30:42.891 UTC [22942] LOG: listening on Unix socket "/tmp/.s.PGSQL.41931"
2026-08-06 18:30:42.920 UTC [22949] LOG: database system was shut down at 2026-08-06 18:30:41 UTC
2026-08-06 18:30:42.924 UTC [22942] LOG: database system is ready to accept connections
I20260806 18:30:43.946177 7439 mini_postgres.cc:96] Postgres bound to 41931
2026-08-06 18:30:43.955 UTC [22969] FATAL: database "slave" does not exist
127.7.67.212:41931 - accepting connections
I20260806 18:30:43.956032 7439 mini_ranger.cc:161] Starting Ranger...
I20260806 18:30:43.973094 7439 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 18:30:44.402268 7439 mini_ranger.cc:87] Created ranger Postgres database
I20260806 18:30:44.402364 7439 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ranger-admin
2026-08-06 18:30:44,775 [I] DB FLAVOR :POSTGRES
2026-08-06 18:30:44,775 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:30:44,775 [I] Checking connection..
2026-08-06 18:30:44,775 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 18:30:45,044 [I] Checking connection passed.
2026-08-06 18:30:45,044 [I] --------- Verifying version history table ---------
2026-08-06 18:30:45,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:30:45,301 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:30:45,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:30:45,546 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:30:45,546 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 18:30:45,546 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 18:30:45.791 UTC [23060] WARNING: there is no transaction in progress
2026-08-06 18:30:45,798 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 18:30:45,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:30:46,051 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 18:30:46,051 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:30:46,051 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 18:30:46,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 18:30:46,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5bn2.c.gcp-upstream.internal','N') ;"
2026-08-06 18:30:46,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:30:47,082 [I] Table x_portal_user does not exist in database ranger
2026-08-06 18:30:47,082 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:30:47,322 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 18:30:47,323 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:30:47,596 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:30:47,863 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 18:30:47,863 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 18:30:48.944 UTC [23227] WARNING: there is no transaction in progress
2026-08-06 18:30:48.960 UTC [23227] WARNING: there is no transaction in progress
2026-08-06 18:30:48.976 UTC [23227] WARNING: there is no transaction in progress
2026-08-06 18:30:48.991 UTC [23227] WARNING: there is no transaction in progress
2026-08-06 18:30:49.007 UTC [23227] WARNING: there is no transaction in progress
2026-08-06 18:30:49.024 UTC [23227] WARNING: there is no transaction in progress
2026-08-06 18:30:49.078 UTC [23227] WARNING: there is no transaction in progress
2026-08-06 18:30:49.085 UTC [23227] WARNING: there is no transaction in progress
2026-08-06 18:30:49.099 UTC [23227] WARNING: there is no transaction in progress
2026-08-06 18:30:49.109 UTC [23227] WARNING: there is no transaction in progress
2026-08-06 18:30:49.120 UTC [23227] WARNING: there is no transaction in progress
2026-08-06 18:30:49.133 UTC [23227] WARNING: there is no transaction in progress
2026-08-06 18:30:49.142 UTC [23227] WARNING: there is no transaction in progress
2026-08-06 18:30:49.149 UTC [23227] WARNING: there is no transaction in progress
2026-08-06 18:30:49.155 UTC [23227] WARNING: there is no transaction in progress
2026-08-06 18:30:49.162 UTC [23227] WARNING: there is no transaction in progress
2026-08-06 18:30:49,768 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 18:30:49,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 18:30:50,001 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 18:30:50,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:30:50,267 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 18:30:50,267 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:30:50,532 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 18:30:50,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:30:50,795 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:30:51,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5bn2.c.gcp-upstream.internal';"
2026-08-06 18:30:51,304 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 18:30:51,305 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 18:30:51,573 [I] DB_PATCHES have already been applied
I20260806 18:30:51.579267 7439 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ranger-admin:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskM6OdK6/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 18:30:51.579339 7439 mini_ranger.cc:193] Using host: 127.7.67.212
I20260806 18:30:51.583917 7439 mini_ranger.cc:227] Ranger admin URL: http://127.7.67.212:58725
Aug 06, 2026 6:30:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:30:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:30:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:30:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:30:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:30:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:30:52 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 18:30:52.781684 7439 mini_ranger.cc:160] Time spent starting Ranger: real 8.826s user 0.000s sys 0.012s
Aug 06, 2026 6:31:12 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:31:12 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 6:31:12 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:31:12 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:31:12 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:31:13 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 18:31:13.911074 7439 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.67.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.67.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.67.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/rangerkms_127.7.67.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.67.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/rangerkms_127.7.67.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.67.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.67.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.67.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/HTTP_127.7.67.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.67.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/HTTP_127.7.67.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 18:31:13 dist-test-slave-5bn2 krb5kdc[22901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041073, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 18:31:13.998883 7439 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 18:31:14.009775 7439 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 18:31:14.788365 7439 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 18:31:14.788492 7439 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ranger-kms
I20260806 18:31:14.788605 7439 mini_ranger_kms.cc:227] Using postgres at 127.7.67.212:41931
2026-08-06 18:31:15,505 [I] DB FLAVOR :POSTGRES
2026-08-06 18:31:15,506 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:31:15,506 [I] Checking connection
2026-08-06 18:31:15,506 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 18:31:15,756 [I] connection success
2026-08-06 18:31:15,756 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 18:31:15,756 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 18:31:15,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 18:31:16,025 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 18:31:16,026 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:31:16,026 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 18:31:16,026 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:41931/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 18:31:16,278 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 18:31:16.760059 7439 mini_ranger_kms.cc:310] Created kms service
I20260806 18:31:16.952606 7439 mini_ranger_kms.cc:326] Created kudu user
I20260806 18:31:16.996846 7439 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 18:31:16 dist-test-slave-5bn2 krb5kdc[22901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041073, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
I20260806 18:31:17.441607 7439 mini_ranger_kms.cc:384] Added ranger policy
I20260806 18:31:17.441772 7439 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ranger-kms:/tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 18:31:17.441797 7439 mini_ranger_kms.cc:240] Using host: 127.7.67.212
I20260806 18:31:17.447417 7439 mini_ranger_kms.cc:276] Ranger KMS PID: 23510
I20260806 18:31:17.447551 7439 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.67.212:46007
Aug 06, 2026 6:31:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:31:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:31:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:31:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:31:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:31:18 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 18:31:19 dist-test-slave-5bn2 krb5kdc[22901](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041079, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.67.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:31:19.882704 7439 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.884s user 0.027s sys 0.353s
I20260806 18:31:19.882874 7439 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 18:31:19.883000 7439 mini_ranger_kms.cc:401] 127.7.67.212:46007/kms/v1/keys
Aug 06 18:31:20 dist-test-slave-5bn2 krb5kdc[22901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041079, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.67.212@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
Aug 06 18:31:20 dist-test-slave-5bn2 krb5kdc[22901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041079, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.67.212@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
Aug 06 18:31:22 dist-test-slave-5bn2 krb5kdc[22901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:31:22.882421 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:36539
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:46331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:36539
--ranger_config_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.212:46007/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:31:23.001677 23591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:31:23.001981 23591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:31:23.002017 23591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:31:23.006173 23591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:31:23.006253 23591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:31:23.006270 23591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:31:23.006285 23591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:31:23.006300 23591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:31:23.012151 23591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:46331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.67.212:46007/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:36539
--ranger_config_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:36539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:31:23.013324 23591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:31:23.014425 23591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:31:23.023983 23603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:31:23.023988 23606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:31:23.023988 23602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:31:23.024490 23591 server_base.cc:1034] running on GCE node
I20260806 18:31:23.025102 23591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:31:23.026307 23591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:31:23.027479 23591 hybrid_clock.cc:648] HybridClock initialized: now 1786041083027457 us; error 40 us; skew 500 ppm
Aug 06 18:31:23 dist-test-slave-5bn2 krb5kdc[22901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041083, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:31:23.033969 23591 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:31:23.034993 23591 webserver.cc:460] Webserver started at http://127.7.67.254:43681/ using document root <none> and password file <none>
I20260806 18:31:23.035594 23591 fs_manager.cc:359] Metadata directory not provided
I20260806 18:31:23.035653 23591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:31:23.035856 23591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:31:23 dist-test-slave-5bn2 krb5kdc[22901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041083, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
I20260806 18:31:23.100433 23591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "ac6f3350b01d4a848d6b00d09a5f8f9d"
format_stamp: "Formatted at 2026-08-06 18:31:23 on dist-test-slave-5bn2"
server_key: "1b7aaf5f0d2bbe840c175c51baa46e44"
server_key_iv: "ca0a1868e00ea5e9f87c59cde5fd72bb"
server_key_version: "kuduclusterkey@0"
I20260806 18:31:23.101294 23591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "ac6f3350b01d4a848d6b00d09a5f8f9d"
format_stamp: "Formatted at 2026-08-06 18:31:23 on dist-test-slave-5bn2"
server_key: "1b7aaf5f0d2bbe840c175c51baa46e44"
server_key_iv: "ca0a1868e00ea5e9f87c59cde5fd72bb"
server_key_version: "kuduclusterkey@0"
I20260806 18:31:23.106528 23591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 18:31:23.125698 23617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:31:23.127492 23591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 18:31:23.127708 23591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "ac6f3350b01d4a848d6b00d09a5f8f9d"
format_stamp: "Formatted at 2026-08-06 18:31:23 on dist-test-slave-5bn2"
server_key: "1b7aaf5f0d2bbe840c175c51baa46e44"
server_key_iv: "ca0a1868e00ea5e9f87c59cde5fd72bb"
server_key_version: "kuduclusterkey@0"
I20260806 18:31:23.127936 23591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:31:23.151935 23591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:31:23.155165 23591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:31:23.155380 23591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:31:23.163626 23591 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:36539
I20260806 18:31:23.163676 23700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:36539 every 8 connection(s)
I20260806 18:31:23.164678 23591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:31:23.167655 23701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:31:23.171918 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 23591
I20260806 18:31:23.172077 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:31:23.172369 7439 external_mini_cluster.cc:1428] Setting key 31508575270194ae263d767b908e446e
I20260806 18:31:23.173784 23701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d: Bootstrap starting.
I20260806 18:31:23.176801 23701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d: Neither blocks nor log segments found. Creating new log.
I20260806 18:31:23.177733 23701 log.cc:824] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:31:23.179936 23701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d: No bootstrap required, opened a new log
Aug 06 18:31:23 dist-test-slave-5bn2 krb5kdc[22901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:31:23.184407 23701 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f3350b01d4a848d6b00d09a5f8f9d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 36539 } }
I20260806 18:31:23.184693 23701 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:31:23.184779 23701 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac6f3350b01d4a848d6b00d09a5f8f9d, State: Initialized, Role: FOLLOWER
I20260806 18:31:23.185300 23701 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f3350b01d4a848d6b00d09a5f8f9d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 36539 } }
I20260806 18:31:23.185462 23701 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:31:23.185552 23701 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:31:23.185668 23701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:31:23.186825 23701 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f3350b01d4a848d6b00d09a5f8f9d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 36539 } }
I20260806 18:31:23.187247 23701 leader_election.cc:302] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac6f3350b01d4a848d6b00d09a5f8f9d; no voters:
I20260806 18:31:23.187631 23701 leader_election.cc:288] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:31:23.189044 23701 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:31:23.189711 23704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:23.173979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44628 (local address 127.7.67.254:36539)
0806 18:31:23.174458 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0806 18:31:23.174479 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:31:23.174525 (+ 46us) server_negotiation.cc:407] Connection header received
0806 18:31:23.175439 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:31:23.175464 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:31:23.175909 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:31:23.176291 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:31:23.177170 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:23.178501 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:23.179838 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:23.180529 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:23.180706 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:31:23.183834 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:31:23.183840 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:31:23.183869 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:31:23.183902 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:31:23.185996 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:31:23.186516 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:31:23.186522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:31:23.186528 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:31:23.186666 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:31:23.187089 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:31:23.187093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:31:23.187095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:31:23.187650 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:31:23.187944 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0806 18:31:23.188221 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 18:31:23.188692 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":130,"threads_started":1}
I20260806 18:31:23.191565 23701 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 18:31:23.192874 23707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:31:23.193085 23707 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d [term 1 LEADER]: Becoming Leader. State: Replica: ac6f3350b01d4a848d6b00d09a5f8f9d, State: Running, Role: LEADER
I20260806 18:31:23.195830 23707 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f3350b01d4a848d6b00d09a5f8f9d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 36539 } }
I20260806 18:31:23.198496 23722 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac6f3350b01d4a848d6b00d09a5f8f9d. Latest consensus state: current_term: 1 leader_uuid: "ac6f3350b01d4a848d6b00d09a5f8f9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f3350b01d4a848d6b00d09a5f8f9d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 36539 } } }
I20260806 18:31:23.198657 23722 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d [sys.catalog]: This master's current role is: LEADER
I20260806 18:31:23.199339 23706 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac6f3350b01d4a848d6b00d09a5f8f9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f3350b01d4a848d6b00d09a5f8f9d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 36539 } } }
I20260806 18:31:23.199456 23706 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d [sys.catalog]: This master's current role is: LEADER
I20260806 18:31:23.203184 23726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:31:23.207088 23726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:31:23.214272 23726 catalog_manager.cc:1347] Generated new cluster ID: 5b61bdefda764b799d2992cf57d4dbe6
I20260806 18:31:23.214421 23726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:31:23.256261 23726 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:31:23.257736 23726 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:31:23.272164 23726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d: Generated new TSK 0
I20260806 18:31:23.273250 23726 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
I20260806 18:31:23.943080 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.212:46007/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:36539
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:46331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:31:24.109882 23767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:31:24.110314 23767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:31:24.110401 23767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:31:24.114629 23767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:31:24.114691 23767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:31:24.114751 23767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:31:24.121984 23767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:46331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.67.212:46007/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:36539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:31:24.123692 23767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:31:24.125072 23767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:31:24.136716 23780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:31:24.136716 23784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:31:24.137089 23778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:31:24.138208 23767 server_base.cc:1034] running on GCE node
I20260806 18:31:24.138684 23767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:31:24.139407 23767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:31:24.140583 23767 hybrid_clock.cc:648] HybridClock initialized: now 1786041084140554 us; error 40 us; skew 500 ppm
Aug 06 18:31:24 dist-test-slave-5bn2 krb5kdc[22901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041084, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:31:24.143539 23767 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:31:24.144541 23767 webserver.cc:460] Webserver started at http://127.7.67.193:42947/ using document root <none> and password file <none>
I20260806 18:31:24.145128 23767 fs_manager.cc:359] Metadata directory not provided
I20260806 18:31:24.145196 23767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:31:24.145426 23767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:31:24 dist-test-slave-5bn2 krb5kdc[22901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041084, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
I20260806 18:31:24.162453 23767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "0e426170b1ba4c3f84e0e1838dfb18ee"
format_stamp: "Formatted at 2026-08-06 18:31:24 on dist-test-slave-5bn2"
server_key: "bacbe3fedcfb795c8503dc104cad5a1b"
server_key_iv: "62d7fddea398c3a23b59c6b50af0df1a"
server_key_version: "kuduclusterkey@0"
I20260806 18:31:24.166324 23767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "0e426170b1ba4c3f84e0e1838dfb18ee"
format_stamp: "Formatted at 2026-08-06 18:31:24 on dist-test-slave-5bn2"
server_key: "bacbe3fedcfb795c8503dc104cad5a1b"
server_key_iv: "62d7fddea398c3a23b59c6b50af0df1a"
server_key_version: "kuduclusterkey@0"
I20260806 18:31:24.177099 23767 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.004s sys 0.000s
I20260806 18:31:24.211392 23793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:31:24.213362 23767 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.002s sys 0.000s
I20260806 18:31:24.213523 23767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "0e426170b1ba4c3f84e0e1838dfb18ee"
format_stamp: "Formatted at 2026-08-06 18:31:24 on dist-test-slave-5bn2"
server_key: "bacbe3fedcfb795c8503dc104cad5a1b"
server_key_iv: "62d7fddea398c3a23b59c6b50af0df1a"
server_key_version: "kuduclusterkey@0"
I20260806 18:31:24.213677 23767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:31:24.242484 23767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:31:24.246296 23767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:31:24.246574 23767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:31:24.247412 23767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:31:24.248569 23767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:31:24.248677 23767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:31:24.248806 23767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:31:24.248888 23767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:31:24.262040 23767 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:45875
I20260806 18:31:24.262089 23952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:45875 every 8 connection(s)
I20260806 18:31:24.263422 23767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:31:24.264591 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 23767
I20260806 18:31:24.264721 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:31:24.265024 7439 external_mini_cluster.cc:1428] Setting key 90e1c9d4f6d15376af29f63a66877031
Aug 06 18:31:24 dist-test-slave-5bn2 krb5kdc[22901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041084, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.194@KRBTEST.COM" created.
I20260806 18:31:24.283543 23955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:24.265632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:39377 (local address 127.7.67.254:36539)
0806 18:31:24.267070 (+ 1438us) server_negotiation.cc:206] Beginning negotiation
0806 18:31:24.267079 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:31:24.268707 (+ 1628us) server_negotiation.cc:407] Connection header received
0806 18:31:24.268757 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:31:24.268760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:31:24.268816 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:31:24.268921 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:31:24.270516 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:24.271498 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:24.273346 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:24.273693 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:24.273771 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:31:24.276993 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:31:24.276997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:31:24.277000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:31:24.277028 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:31:24.278894 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:31:24.279532 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:31:24.279535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:31:24.279537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:31:24.279637 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:31:24.280059 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:31:24.280063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:31:24.280064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:31:24.280306 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:31:24.280408 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:31:24.283163 (+ 2755us) server_negotiation.cc:299] Negotiation successful
0806 18:31:24.283323 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1349,"thread_start_us":111,"threads_started":1}
I20260806 18:31:24.285494 23956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:24.266204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:36539 (local address 127.7.67.193:39377)
0806 18:31:24.267357 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0806 18:31:24.267416 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 18:31:24.268406 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:31:24.269248 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:31:24.269258 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:31:24.269634 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:31:24.270344 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:31:24.270364 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:31:24.271659 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:31:24.271662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:31:24.273210 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:31:24.273221 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:31:24.274012 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:31:24.274016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:31:24.274224 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:31:24.274963 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:31:24.274983 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:31:24.276774 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:31:24.279033 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:31:24.279040 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:31:24.279058 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:31:24.279409 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:31:24.279749 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:31:24.279753 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:31:24.279755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:31:24.279950 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:31:24.280430 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:31:24.280437 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:31:24.280806 (+ 369us) client_negotiation.cc:769] Sending connection context
0806 18:31:24.281038 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 18:31:24.284335 (+ 3297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":920,"thread_start_us":157,"threads_started":1}
I20260806 18:31:24.287297 23953 heartbeater.cc:347] Connected to a master server at 127.7.67.254:36539
I20260806 18:31:24.287561 23953 heartbeater.cc:464] Registering TS with master...
I20260806 18:31:24.288172 23953 heartbeater.cc:511] Master 127.7.67.254:36539 requested a full tablet report, sending...
I20260806 18:31:24.290200 23648 ts_manager.cc:194] Registered new tserver with Master: 0e426170b1ba4c3f84e0e1838dfb18ee (127.7.67.193:45875)
I20260806 18:31:24.291973 23648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:39377
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
I20260806 18:31:24.300011 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.212:46007/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:36539
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:46331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:31:24 dist-test-slave-5bn2 krb5kdc[22901](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041084, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 18:31:24.468219 23959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:31:24.468727 23959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:31:24.468856 23959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:31:24.475117 23959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:31:24.475215 23959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:31:24.475310 23959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:31:24.483484 23959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:46331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.67.212:46007/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:36539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:31:24.485199 23959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:31:24.486732 23959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:31:24.500901 23974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:31:24.501447 23959 server_base.cc:1034] running on GCE node
W20260806 18:31:24.501156 23978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:31:24.501181 23972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:31:24.502326 23959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:31:24.503211 23959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:31:24.506477 23959 hybrid_clock.cc:648] HybridClock initialized: now 1786041084506451 us; error 38 us; skew 500 ppm
Aug 06 18:31:24 dist-test-slave-5bn2 krb5kdc[22901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041084, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:31:24.509493 23959 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:31:24.510603 23959 webserver.cc:460] Webserver started at http://127.7.67.194:35013/ using document root <none> and password file <none>
I20260806 18:31:24.511240 23959 fs_manager.cc:359] Metadata directory not provided
I20260806 18:31:24.511313 23959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:31:24.511567 23959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:31:24 dist-test-slave-5bn2 krb5kdc[22901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041084, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
I20260806 18:31:24.531406 23959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "36151543a1b447698a197e78e2adab12"
format_stamp: "Formatted at 2026-08-06 18:31:24 on dist-test-slave-5bn2"
server_key: "83c987adccc32938c43256e6417a4fa0"
server_key_iv: "cf4313ab86f70e7f06cf133effe557af"
server_key_version: "kuduclusterkey@0"
I20260806 18:31:24.532944 23959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "36151543a1b447698a197e78e2adab12"
format_stamp: "Formatted at 2026-08-06 18:31:24 on dist-test-slave-5bn2"
server_key: "83c987adccc32938c43256e6417a4fa0"
server_key_iv: "cf4313ab86f70e7f06cf133effe557af"
server_key_version: "kuduclusterkey@0"
I20260806 18:31:24.539094 23959 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260806 18:31:24.563133 23988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:31:24.566601 23959 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.001s
I20260806 18:31:24.566915 23959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "36151543a1b447698a197e78e2adab12"
format_stamp: "Formatted at 2026-08-06 18:31:24 on dist-test-slave-5bn2"
server_key: "83c987adccc32938c43256e6417a4fa0"
server_key_iv: "cf4313ab86f70e7f06cf133effe557af"
server_key_version: "kuduclusterkey@0"
I20260806 18:31:24.567198 23959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:31:24.592104 23959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:31:24.595729 23959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:31:24.595937 23959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:31:24.596657 23959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:31:24.597765 23959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:31:24.597827 23959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:31:24.597882 23959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:31:24.597909 23959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:31:24.609138 23959 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:42915
I20260806 18:31:24.609198 24148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:42915 every 8 connection(s)
I20260806 18:31:24.610404 23959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:31:24.612102 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 23959
I20260806 18:31:24.612231 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:31:24.612505 7439 external_mini_cluster.cc:1428] Setting key a9e3ad87e6e90312ee187ccc6b50658a
Aug 06 18:31:24 dist-test-slave-5bn2 krb5kdc[22901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041084, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.195@KRBTEST.COM; defaulting to no policy
I20260806 18:31:24.627527 23955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:24.613051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:38889 (local address 127.7.67.254:36539)
0806 18:31:24.613203 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:31:24.613208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:31:24.613785 (+ 577us) server_negotiation.cc:407] Connection header received
0806 18:31:24.615993 (+ 2208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:31:24.615997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:31:24.616053 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:31:24.616148 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:31:24.617440 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:24.617975 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:24.619425 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:24.619786 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:24.619939 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:31:24.622886 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:31:24.622891 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:31:24.622895 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:31:24.622930 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:31:24.624759 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:31:24.625481 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:31:24.625486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:31:24.625488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:31:24.625600 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:31:24.626048 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:31:24.626051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:31:24.626053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:31:24.626324 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:31:24.626444 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 18:31:24.627119 (+ 675us) server_negotiation.cc:299] Negotiation successful
0806 18:31:24.627316 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 18:31:24.628305 24151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:24.613058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:36539 (local address 127.7.67.194:38889)
0806 18:31:24.613629 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0806 18:31:24.613692 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 18:31:24.615718 (+ 2026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:31:24.616376 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:31:24.616387 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:31:24.616737 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:31:24.617274 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:31:24.617289 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:31:24.618106 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:31:24.618109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:31:24.619243 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:31:24.619256 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:31:24.619974 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:31:24.619978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:31:24.620183 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:31:24.620935 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:31:24.620960 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:31:24.622545 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:31:24.624913 (+ 2368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:31:24.624922 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:31:24.624946 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:31:24.625338 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:31:24.625712 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:31:24.625716 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:31:24.625718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:31:24.625941 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:31:24.626489 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:31:24.626498 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:31:24.626813 (+ 315us) client_negotiation.cc:769] Sending connection context
0806 18:31:24.627045 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 18:31:24.627373 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":160,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.195@KRBTEST.COM" created.
I20260806 18:31:24.629925 24149 heartbeater.cc:347] Connected to a master server at 127.7.67.254:36539
I20260806 18:31:24.630194 24149 heartbeater.cc:464] Registering TS with master...
I20260806 18:31:24.630785 24149 heartbeater.cc:511] Master 127.7.67.254:36539 requested a full tablet report, sending...
I20260806 18:31:24.632131 23648 ts_manager.cc:194] Registered new tserver with Master: 36151543a1b447698a197e78e2adab12 (127.7.67.194:42915)
I20260806 18:31:24.632969 23648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:38889
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
I20260806 18:31:24.658345 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.212:46007/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:36539
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:46331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:31:24 dist-test-slave-5bn2 krb5kdc[22901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041084, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
W20260806 18:31:24.826546 24155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:31:24.827057 24155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:31:24.827167 24155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:31:24.833082 24155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:31:24.833268 24155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:31:24.833405 24155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:31:24.841053 24155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:46331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.67.212:46007/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:36539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:31:24.842911 24155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:31:24.844223 24155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:31:24.857473 24164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:31:24.857800 24155 server_base.cc:1034] running on GCE node
W20260806 18:31:24.857589 24167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:31:24.858381 24171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:31:24.858959 24155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:31:24.859719 24155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:31:24.860882 24155 hybrid_clock.cc:648] HybridClock initialized: now 1786041084860819 us; error 77 us; skew 500 ppm
Aug 06 18:31:24 dist-test-slave-5bn2 krb5kdc[22901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041084, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:31:24.864567 24155 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:31:24.865629 24155 webserver.cc:460] Webserver started at http://127.7.67.195:36409/ using document root <none> and password file <none>
I20260806 18:31:24.866312 24155 fs_manager.cc:359] Metadata directory not provided
I20260806 18:31:24.866379 24155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:31:24.866606 24155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:31:24 dist-test-slave-5bn2 krb5kdc[22901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041084, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
I20260806 18:31:24.884228 24155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "f9070087de924c4180558fd59b8185fa"
format_stamp: "Formatted at 2026-08-06 18:31:24 on dist-test-slave-5bn2"
server_key: "0d249c6a14a7b1d968856057f4360032"
server_key_iv: "6697f38e3054a12be4a0e87d5b282da8"
server_key_version: "kuduclusterkey@0"
I20260806 18:31:24.886726 24155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "f9070087de924c4180558fd59b8185fa"
format_stamp: "Formatted at 2026-08-06 18:31:24 on dist-test-slave-5bn2"
server_key: "0d249c6a14a7b1d968856057f4360032"
server_key_iv: "6697f38e3054a12be4a0e87d5b282da8"
server_key_version: "kuduclusterkey@0"
I20260806 18:31:24.892486 24155 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 18:31:24.909830 24180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:31:24.912005 24155 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 18:31:24.912151 24155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "f9070087de924c4180558fd59b8185fa"
format_stamp: "Formatted at 2026-08-06 18:31:24 on dist-test-slave-5bn2"
server_key: "0d249c6a14a7b1d968856057f4360032"
server_key_iv: "6697f38e3054a12be4a0e87d5b282da8"
server_key_version: "kuduclusterkey@0"
I20260806 18:31:24.912299 24155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:31:24.930426 24155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:31:24.934234 24155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:31:24.934490 24155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:31:24.935256 24155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:31:24.936373 24155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:31:24.936478 24155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:31:24.936591 24155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:31:24.936656 24155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:31:24.951099 24155 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:46711
I20260806 18:31:24.952378 24155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:31:24.955018 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 24155
I20260806 18:31:24.955159 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:31:24.955492 7439 external_mini_cluster.cc:1428] Setting key 270eb6403e8d9bf342af4a7dde1c2a18
I20260806 18:31:24.955011 24352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:46711 every 8 connection(s)
Aug 06 18:31:24 dist-test-slave-5bn2 krb5kdc[22901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041084, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:31:24.971802 23955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:24.957844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:43363 (local address 127.7.67.254:36539)
0806 18:31:24.957970 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:31:24.957975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:31:24.958592 (+ 617us) server_negotiation.cc:407] Connection header received
0806 18:31:24.959879 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:31:24.959883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:31:24.959937 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:31:24.959989 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:31:24.961601 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:24.962359 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:24.964121 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:24.964472 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:24.964554 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:31:24.967604 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:31:24.967608 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:31:24.967610 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:31:24.967640 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:31:24.969286 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:31:24.969917 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:31:24.969921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:31:24.969923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:31:24.970010 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:31:24.970423 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:31:24.970426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:31:24.970428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:31:24.970663 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:31:24.970741 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 18:31:24.971481 (+ 740us) server_negotiation.cc:299] Negotiation successful
0806 18:31:24.971634 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:31:24.973459 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:24.956728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:36539 (local address 127.7.67.195:43363)
0806 18:31:24.958429 (+ 1701us) negotiation.cc:107] Waiting for socket to connect
0806 18:31:24.958485 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 18:31:24.959579 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:31:24.960401 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:31:24.960410 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:31:24.960771 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:31:24.961423 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:31:24.961441 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:31:24.962521 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:31:24.962525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:31:24.963982 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:31:24.963994 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:31:24.964761 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:31:24.964764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:31:24.964966 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:31:24.965678 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:31:24.965699 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:31:24.967395 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:31:24.969427 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:31:24.969434 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:31:24.969451 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:31:24.969802 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:31:24.970135 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:31:24.970138 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:31:24.970140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:31:24.970321 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:31:24.970811 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:31:24.970818 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:31:24.971197 (+ 379us) client_negotiation.cc:769] Sending connection context
0806 18:31:24.972023 (+ 826us) client_negotiation.cc:240] Negotiation successful
0806 18:31:24.972384 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1482,"thread_start_us":132,"threads_started":1}
I20260806 18:31:24.974905 24354 heartbeater.cc:347] Connected to a master server at 127.7.67.254:36539
I20260806 18:31:24.975136 24354 heartbeater.cc:464] Registering TS with master...
I20260806 18:31:24.975793 24354 heartbeater.cc:511] Master 127.7.67.254:36539 requested a full tablet report, sending...
I20260806 18:31:24.977236 23648 ts_manager.cc:194] Registered new tserver with Master: f9070087de924c4180558fd59b8185fa (127.7.67.195:46711)
I20260806 18:31:24.978041 23648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:43363
I20260806 18:31:24.981848 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:31:24.997720 23955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:24.987372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44638 (local address 127.7.67.254:36539)
0806 18:31:24.987515 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:31:24.987519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:31:24.988167 (+ 648us) server_negotiation.cc:407] Connection header received
0806 18:31:24.988354 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:31:24.988358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:31:24.988408 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:31:24.988471 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:31:24.989529 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:24.990396 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:24.991740 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:24.992085 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:24.992158 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:31:24.993996 (+ 1838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:31:24.994001 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:31:24.994003 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:31:24.994033 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:31:24.995783 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:31:24.996300 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:31:24.996304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:31:24.996306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:31:24.996397 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:31:24.996710 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:31:24.996714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:31:24.996716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:31:24.996940 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:31:24.997049 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 18:31:24.997404 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 18:31:24.997545 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:31:25.002063 23648 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:31:25.006758 23648 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:31:25.033028 24372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:25.020516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:45875 (local address 127.0.0.1:46126)
0806 18:31:25.024205 (+ 3689us) negotiation.cc:107] Waiting for socket to connect
0806 18:31:25.024261 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 18:31:25.024407 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:31:25.026264 (+ 1857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:31:25.026271 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:31:25.026293 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:31:25.026456 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:31:25.026462 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.028770 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.028773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:31:25.030190 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:31:25.030202 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.032415 (+ 2213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.032422 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:31:25.032564 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:31:25.032611 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 18:31:25.032725 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 18:31:25.032862 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3506,"thread_start_us":120,"threads_started":1}
I20260806 18:31:25.033993 24371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:25.021384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46126 (local address 127.7.67.193:45875)
0806 18:31:25.023943 (+ 2559us) server_negotiation.cc:206] Beginning negotiation
0806 18:31:25.023951 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:31:25.025216 (+ 1265us) server_negotiation.cc:407] Connection header received
0806 18:31:25.025724 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:31:25.025731 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:31:25.025957 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:31:25.026094 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:31:25.026584 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.028648 (+ 2064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.030334 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.031191 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.031289 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:31:25.031468 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 18:31:25.033731 (+ 2263us) server_negotiation.cc:299] Negotiation successful
0806 18:31:25.033846 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2435,"thread_start_us":106,"threads_started":1}
I20260806 18:31:25.036473 24376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:25.023726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:46711 (local address 127.0.0.1:46424)
0806 18:31:25.031028 (+ 7302us) negotiation.cc:107] Waiting for socket to connect
0806 18:31:25.031045 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:31:25.031142 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:31:25.032021 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:31:25.032024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:31:25.032040 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:31:25.032139 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:31:25.032146 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.033700 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.033703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:31:25.034803 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:31:25.034814 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.036084 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.036087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:31:25.036174 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:31:25.036199 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:31:25.036266 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 18:31:25.036339 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7225,"thread_start_us":89,"threads_started":1}
I20260806 18:31:25.037721 23864 tablet_service.cc:1467] Processing CreateTablet for tablet 20820271755d454d9fc55e65becb9c4b (DEFAULT_TABLE table=test-table [id=0806977650f349dd82196c3aa3648aff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:31:25.039975 23864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20820271755d454d9fc55e65becb9c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:31:25.040047 24378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:25.024139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46424 (local address 127.7.67.195:46711)
0806 18:31:25.031422 (+ 7283us) server_negotiation.cc:206] Beginning negotiation
0806 18:31:25.031435 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:31:25.031461 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:31:25.031545 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:31:25.031552 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:31:25.031790 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:31:25.032243 (+ 453us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:31:25.032396 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.033550 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.035006 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.035759 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.035862 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:31:25.036018 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 18:31:25.039761 (+ 3743us) server_negotiation.cc:299] Negotiation successful
0806 18:31:25.039881 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7162,"thread_start_us":155,"threads_started":1}
I20260806 18:31:25.050328 24259 tablet_service.cc:1467] Processing CreateTablet for tablet 20820271755d454d9fc55e65becb9c4b (DEFAULT_TABLE table=test-table [id=0806977650f349dd82196c3aa3648aff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:31:25.051751 24259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20820271755d454d9fc55e65becb9c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:31:25.056943 24379 tablet_bootstrap.cc:492] T 20820271755d454d9fc55e65becb9c4b P 0e426170b1ba4c3f84e0e1838dfb18ee: Bootstrap starting.
I20260806 18:31:25.059653 24379 tablet_bootstrap.cc:654] T 20820271755d454d9fc55e65becb9c4b P 0e426170b1ba4c3f84e0e1838dfb18ee: Neither blocks nor log segments found. Creating new log.
I20260806 18:31:25.060698 24379 log.cc:824] T 20820271755d454d9fc55e65becb9c4b P 0e426170b1ba4c3f84e0e1838dfb18ee: Log is configured to *not* fsync() on all Append() calls
I20260806 18:31:25.063341 24379 tablet_bootstrap.cc:492] T 20820271755d454d9fc55e65becb9c4b P 0e426170b1ba4c3f84e0e1838dfb18ee: No bootstrap required, opened a new log
I20260806 18:31:25.063525 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:25.023026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:42915 (local address 127.0.0.1:45808)
0806 18:31:25.025374 (+ 2348us) negotiation.cc:107] Waiting for socket to connect
0806 18:31:25.025386 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:31:25.025486 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:31:25.038081 (+ 12595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:31:25.038085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:31:25.038109 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:31:25.038201 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:31:25.038207 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.060591 (+ 22384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.060595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:31:25.061780 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:31:25.061798 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.063074 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.063079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:31:25.063185 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:31:25.063220 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 18:31:25.063278 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 18:31:25.063368 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2282,"thread_start_us":93,"threads_started":1}
I20260806 18:31:25.063592 24379 ts_tablet_manager.cc:1397] T 20820271755d454d9fc55e65becb9c4b P 0e426170b1ba4c3f84e0e1838dfb18ee: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 18:31:25.064327 24375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:25.023230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45808 (local address 127.7.67.194:42915)
0806 18:31:25.037091 (+ 13861us) server_negotiation.cc:206] Beginning negotiation
0806 18:31:25.037105 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:31:25.037132 (+ 27us) server_negotiation.cc:407] Connection header received
0806 18:31:25.037212 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:31:25.037217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:31:25.037425 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:31:25.038369 (+ 944us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:31:25.038523 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.039606 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.062030 (+ 22424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.062897 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.063848 (+ 951us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:31:25.064026 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 18:31:25.064086 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 18:31:25.064182 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13740,"thread_start_us":138,"threads_started":1}
I20260806 18:31:25.067646 24037 tablet_service.cc:1467] Processing CreateTablet for tablet 20820271755d454d9fc55e65becb9c4b (DEFAULT_TABLE table=test-table [id=0806977650f349dd82196c3aa3648aff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:31:25.068967 24037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20820271755d454d9fc55e65becb9c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:31:25.069947 24379 raft_consensus.cc:348] T 20820271755d454d9fc55e65becb9c4b P 0e426170b1ba4c3f84e0e1838dfb18ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e426170b1ba4c3f84e0e1838dfb18ee" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45875 } } peers { permanent_uuid: "36151543a1b447698a197e78e2adab12" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42915 } } peers { permanent_uuid: "f9070087de924c4180558fd59b8185fa" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46711 } }
I20260806 18:31:25.070276 24379 raft_consensus.cc:374] T 20820271755d454d9fc55e65becb9c4b P 0e426170b1ba4c3f84e0e1838dfb18ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:31:25.070343 24379 raft_consensus.cc:729] T 20820271755d454d9fc55e65becb9c4b P 0e426170b1ba4c3f84e0e1838dfb18ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e426170b1ba4c3f84e0e1838dfb18ee, State: Initialized, Role: FOLLOWER
I20260806 18:31:25.070943 24379 consensus_queue.cc:260] T 20820271755d454d9fc55e65becb9c4b P 0e426170b1ba4c3f84e0e1838dfb18ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e426170b1ba4c3f84e0e1838dfb18ee" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45875 } } peers { permanent_uuid: "36151543a1b447698a197e78e2adab12" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42915 } } peers { permanent_uuid: "f9070087de924c4180558fd59b8185fa" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46711 } }
I20260806 18:31:25.072126 24379 ts_tablet_manager.cc:1428] T 20820271755d454d9fc55e65becb9c4b P 0e426170b1ba4c3f84e0e1838dfb18ee: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260806 18:31:25.072651 23953 heartbeater.cc:503] Master 127.7.67.254:36539 was elected leader, sending a full tablet report...
I20260806 18:31:25.079540 24380 tablet_bootstrap.cc:492] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa: Bootstrap starting.
I20260806 18:31:25.085801 24380 tablet_bootstrap.cc:654] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa: Neither blocks nor log segments found. Creating new log.
I20260806 18:31:25.090714 24380 log.cc:824] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa: Log is configured to *not* fsync() on all Append() calls
I20260806 18:31:25.091708 24385 tablet_bootstrap.cc:492] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12: Bootstrap starting.
I20260806 18:31:25.094527 24385 tablet_bootstrap.cc:654] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12: Neither blocks nor log segments found. Creating new log.
I20260806 18:31:25.100391 24385 log.cc:824] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12: Log is configured to *not* fsync() on all Append() calls
I20260806 18:31:25.105315 24380 tablet_bootstrap.cc:492] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa: No bootstrap required, opened a new log
I20260806 18:31:25.105607 24380 ts_tablet_manager.cc:1397] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa: Time spent bootstrapping tablet: real 0.026s user 0.008s sys 0.000s
I20260806 18:31:25.110000 24380 raft_consensus.cc:348] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e426170b1ba4c3f84e0e1838dfb18ee" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45875 } } peers { permanent_uuid: "36151543a1b447698a197e78e2adab12" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42915 } } peers { permanent_uuid: "f9070087de924c4180558fd59b8185fa" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46711 } }
I20260806 18:31:25.110347 24380 raft_consensus.cc:374] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:31:25.110419 24380 raft_consensus.cc:729] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9070087de924c4180558fd59b8185fa, State: Initialized, Role: FOLLOWER
I20260806 18:31:25.111264 24380 consensus_queue.cc:260] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e426170b1ba4c3f84e0e1838dfb18ee" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45875 } } peers { permanent_uuid: "36151543a1b447698a197e78e2adab12" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42915 } } peers { permanent_uuid: "f9070087de924c4180558fd59b8185fa" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46711 } }
I20260806 18:31:25.112560 24380 ts_tablet_manager.cc:1428] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260806 18:31:25.112586 24385 tablet_bootstrap.cc:492] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12: No bootstrap required, opened a new log
I20260806 18:31:25.112819 24385 ts_tablet_manager.cc:1397] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.000s
I20260806 18:31:25.113031 24354 heartbeater.cc:503] Master 127.7.67.254:36539 was elected leader, sending a full tablet report...
I20260806 18:31:25.116834 24385 raft_consensus.cc:348] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e426170b1ba4c3f84e0e1838dfb18ee" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45875 } } peers { permanent_uuid: "36151543a1b447698a197e78e2adab12" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42915 } } peers { permanent_uuid: "f9070087de924c4180558fd59b8185fa" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46711 } }
I20260806 18:31:25.117213 24385 raft_consensus.cc:374] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:31:25.117306 24385 raft_consensus.cc:729] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36151543a1b447698a197e78e2adab12, State: Initialized, Role: FOLLOWER
I20260806 18:31:25.117934 24385 consensus_queue.cc:260] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e426170b1ba4c3f84e0e1838dfb18ee" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45875 } } peers { permanent_uuid: "36151543a1b447698a197e78e2adab12" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42915 } } peers { permanent_uuid: "f9070087de924c4180558fd59b8185fa" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46711 } }
I20260806 18:31:25.119122 24385 ts_tablet_manager.cc:1428] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:31:25.119525 24149 heartbeater.cc:503] Master 127.7.67.254:36539 was elected leader, sending a full tablet report...
Aug 06 18:31:25 dist-test-slave-5bn2 krb5kdc[22901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041084, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
I20260806 18:31:25.155191 24397 raft_consensus.cc:482] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:31:25.155463 24397 raft_consensus.cc:504] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e426170b1ba4c3f84e0e1838dfb18ee" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45875 } } peers { permanent_uuid: "36151543a1b447698a197e78e2adab12" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42915 } } peers { permanent_uuid: "f9070087de924c4180558fd59b8185fa" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46711 } }
I20260806 18:31:25.156991 24397 leader_election.cc:288] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e426170b1ba4c3f84e0e1838dfb18ee (127.7.67.193:45875), f9070087de924c4180558fd59b8185fa (127.7.67.195:46711)
I20260806 18:31:25.173926 24371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:25.160442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:46441 (local address 127.7.67.193:45875)
0806 18:31:25.160562 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:31:25.160566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:31:25.161190 (+ 624us) server_negotiation.cc:407] Connection header received
0806 18:31:25.161489 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:31:25.161494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:31:25.161556 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:31:25.161609 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:31:25.162059 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.162769 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.172460 (+ 9691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.173125 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.173236 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:31:25.173296 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 18:31:25.173685 (+ 389us) server_negotiation.cc:299] Negotiation successful
0806 18:31:25.173776 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:31:25.174355 24399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:25.157427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:45875 (local address 127.7.67.194:46441)
0806 18:31:25.161074 (+ 3647us) negotiation.cc:107] Waiting for socket to connect
0806 18:31:25.161098 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 18:31:25.161324 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:31:25.161764 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:31:25.161767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:31:25.161787 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:31:25.161924 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:31:25.161933 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.162985 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.162988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:31:25.172268 (+ 9280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:31:25.172282 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.173405 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.173408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:31:25.173545 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:31:25.173571 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:31:25.174102 (+ 531us) client_negotiation.cc:240] Negotiation successful
0806 18:31:25.174193 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3514,"mutex_wait_us":25,"thread_start_us":89,"threads_started":1}
I20260806 18:31:25.175298 23889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20820271755d454d9fc55e65becb9c4b" candidate_uuid: "36151543a1b447698a197e78e2adab12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e426170b1ba4c3f84e0e1838dfb18ee" is_pre_election: true
I20260806 18:31:25.175673 23889 raft_consensus.cc:2454] T 20820271755d454d9fc55e65becb9c4b P 0e426170b1ba4c3f84e0e1838dfb18ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36151543a1b447698a197e78e2adab12 in term 0.
I20260806 18:31:25.176496 23989 leader_election.cc:302] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e426170b1ba4c3f84e0e1838dfb18ee, 36151543a1b447698a197e78e2adab12; no voters:
I20260806 18:31:25.176821 24397 raft_consensus.cc:2781] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:31:25.176901 24397 raft_consensus.cc:482] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:31:25.176936 24397 raft_consensus.cc:3037] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:31:25.178303 24397 raft_consensus.cc:504] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e426170b1ba4c3f84e0e1838dfb18ee" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45875 } } peers { permanent_uuid: "36151543a1b447698a197e78e2adab12" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42915 } } peers { permanent_uuid: "f9070087de924c4180558fd59b8185fa" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46711 } }
I20260806 18:31:25.178694 24397 leader_election.cc:288] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [CANDIDATE]: Term 1 election: Requested vote from peers 0e426170b1ba4c3f84e0e1838dfb18ee (127.7.67.193:45875), f9070087de924c4180558fd59b8185fa (127.7.67.195:46711)
I20260806 18:31:25.179385 23889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20820271755d454d9fc55e65becb9c4b" candidate_uuid: "36151543a1b447698a197e78e2adab12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e426170b1ba4c3f84e0e1838dfb18ee"
I20260806 18:31:25.179518 23889 raft_consensus.cc:3037] T 20820271755d454d9fc55e65becb9c4b P 0e426170b1ba4c3f84e0e1838dfb18ee [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:31:25.180450 24378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:25.160218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:58977 (local address 127.7.67.195:46711)
0806 18:31:25.160359 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:31:25.160364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:31:25.171238 (+ 10874us) server_negotiation.cc:407] Connection header received
0806 18:31:25.171378 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:31:25.171382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:31:25.171440 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:31:25.171508 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:31:25.173916 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.174621 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.179125 (+ 4504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.179721 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.179847 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:31:25.179904 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:31:25.180226 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 18:31:25.180310 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:31:25.180840 24398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:25.157848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:46711 (local address 127.7.67.194:58977)
0806 18:31:25.171134 (+ 13286us) negotiation.cc:107] Waiting for socket to connect
0806 18:31:25.171154 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 18:31:25.171282 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:31:25.173318 (+ 2036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:31:25.173322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:31:25.173351 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:31:25.173494 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:31:25.173501 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.175155 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.175159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:31:25.176225 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:31:25.176237 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.179978 (+ 3741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.179982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:31:25.180083 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:31:25.180107 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:31:25.180595 (+ 488us) client_negotiation.cc:240] Negotiation successful
0806 18:31:25.180681 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13184,"thread_start_us":127,"threads_started":1}
I20260806 18:31:25.180907 23889 raft_consensus.cc:2454] T 20820271755d454d9fc55e65becb9c4b P 0e426170b1ba4c3f84e0e1838dfb18ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36151543a1b447698a197e78e2adab12 in term 1.
I20260806 18:31:25.181672 23989 leader_election.cc:302] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e426170b1ba4c3f84e0e1838dfb18ee, 36151543a1b447698a197e78e2adab12; no voters:
I20260806 18:31:25.181789 24283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20820271755d454d9fc55e65becb9c4b" candidate_uuid: "36151543a1b447698a197e78e2adab12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9070087de924c4180558fd59b8185fa" is_pre_election: true
I20260806 18:31:25.182088 24281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20820271755d454d9fc55e65becb9c4b" candidate_uuid: "36151543a1b447698a197e78e2adab12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9070087de924c4180558fd59b8185fa"
I20260806 18:31:25.182128 24283 raft_consensus.cc:2454] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36151543a1b447698a197e78e2adab12 in term 0.
I20260806 18:31:25.182948 24397 raft_consensus.cc:2781] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:31:25.183207 24397 raft_consensus.cc:686] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [term 1 LEADER]: Becoming Leader. State: Replica: 36151543a1b447698a197e78e2adab12, State: Running, Role: LEADER
I20260806 18:31:25.183563 24397 consensus_queue.cc:237] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e426170b1ba4c3f84e0e1838dfb18ee" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45875 } } peers { permanent_uuid: "36151543a1b447698a197e78e2adab12" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42915 } } peers { permanent_uuid: "f9070087de924c4180558fd59b8185fa" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46711 } }
I20260806 18:31:25.187197 23648 catalog_manager.cc:5495] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36151543a1b447698a197e78e2adab12 (127.7.67.194). New cstate: current_term: 1 leader_uuid: "36151543a1b447698a197e78e2adab12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e426170b1ba4c3f84e0e1838dfb18ee" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36151543a1b447698a197e78e2adab12" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9070087de924c4180558fd59b8185fa" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46711 } health_report { overall_health: UNKNOWN } } }
W20260806 18:31:25.204607 24355 tablet.cc:2369] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:31:25.235795 24375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:25.230242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45816 (local address 127.7.67.194:42915)
0806 18:31:25.230409 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 18:31:25.230414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:31:25.230432 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:31:25.230533 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:31:25.230536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:31:25.230605 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:31:25.230727 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:31:25.231036 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.231794 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.233040 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:31:25.233370 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:31:25.233467 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:31:25.233913 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:31:25.234516 (+ 603us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:31:25.234633 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 18:31:25.235559 (+ 926us) server_negotiation.cc:299] Negotiation successful
0806 18:31:25.235632 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:31:25.243469 24281 raft_consensus.cc:3037] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:31:25.245023 24281 raft_consensus.cc:1261] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa [term 1 FOLLOWER]: Refusing update from remote peer 36151543a1b447698a197e78e2adab12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:31:25.245837 24397 consensus_queue.cc:1035] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9070087de924c4180558fd59b8185fa" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:31:25.255119 23889 raft_consensus.cc:1261] T 20820271755d454d9fc55e65becb9c4b P 0e426170b1ba4c3f84e0e1838dfb18ee [term 1 FOLLOWER]: Refusing update from remote peer 36151543a1b447698a197e78e2adab12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:31:25.256003 24397 consensus_queue.cc:1035] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e426170b1ba4c3f84e0e1838dfb18ee" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:31:25.266153 23954 tablet.cc:2369] T 20820271755d454d9fc55e65becb9c4b P 0e426170b1ba4c3f84e0e1838dfb18ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:31:25.270516 24407 mvcc.cc:205] Tried to move back new op lower bound from 7315624285148270592 to 7315624284915884032. Current Snapshot: MvccSnapshot[applied={T|T < 7315624285148270592}]
I20260806 18:31:25.278941 24410 mvcc.cc:205] Tried to move back new op lower bound from 7315624285148270592 to 7315624284915884032. Current Snapshot: MvccSnapshot[applied={T|T < 7315624285148270592}]
I20260806 18:31:25.280462 24408 mvcc.cc:205] Tried to move back new op lower bound from 7315624285148270592 to 7315624284915884032. Current Snapshot: MvccSnapshot[applied={T|T < 7315624285148270592}]
I20260806 18:31:25.291131 23647 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:31:25.291397 23647 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:31:25.294694 23647 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac6f3350b01d4a848d6b00d09a5f8f9d: Sending DeleteTablet for 3 replicas of tablet 20820271755d454d9fc55e65becb9c4b
I20260806 18:31:25.295689 24037 tablet_service.cc:1514] Processing DeleteTablet for tablet 20820271755d454d9fc55e65becb9c4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:31:25 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:45808
I20260806 18:31:25.295862 23864 tablet_service.cc:1514] Processing DeleteTablet for tablet 20820271755d454d9fc55e65becb9c4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:31:25 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:46126
I20260806 18:31:25.296418 24259 tablet_service.cc:1514] Processing DeleteTablet for tablet 20820271755d454d9fc55e65becb9c4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:31:25 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:46424
I20260806 18:31:25.297397 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 23767
I20260806 18:31:25.303648 24425 tablet_replica.cc:330] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa: stopping tablet replica
I20260806 18:31:25.304121 24425 raft_consensus.cc:2229] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:31:25.304471 24425 raft_consensus.cc:2258] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:31:25.305716 24424 tablet_replica.cc:330] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12: stopping tablet replica
I20260806 18:31:25.306079 24424 raft_consensus.cc:2229] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:31:25.306552 24424 raft_consensus.cc:2258] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:31:25.316476 24424 ts_tablet_manager.cc:1905] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:31:25.319445 24425 ts_tablet_manager.cc:1905] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:31:25.322376 24425 ts_tablet_manager.cc:1918] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:31:25.322492 24425 log.cc:1196] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/20820271755d454d9fc55e65becb9c4b
I20260806 18:31:25.323235 24425 ts_tablet_manager.cc:1939] T 20820271755d454d9fc55e65becb9c4b P f9070087de924c4180558fd59b8185fa: Deleting consensus metadata
I20260806 18:31:25.324775 23622 catalog_manager.cc:4841] TS f9070087de924c4180558fd59b8185fa (127.7.67.195:46711): tablet 20820271755d454d9fc55e65becb9c4b (table test-table [id=0806977650f349dd82196c3aa3648aff]) successfully deleted
I20260806 18:31:25.327005 24424 ts_tablet_manager.cc:1918] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:31:25.327121 24424 log.cc:1196] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/20820271755d454d9fc55e65becb9c4b
I20260806 18:31:25.327461 24424 ts_tablet_manager.cc:1939] T 20820271755d454d9fc55e65becb9c4b P 36151543a1b447698a197e78e2adab12: Deleting consensus metadata
I20260806 18:31:25.328846 23621 catalog_manager.cc:4841] TS 36151543a1b447698a197e78e2adab12 (127.7.67.194:42915): tablet 20820271755d454d9fc55e65becb9c4b (table test-table [id=0806977650f349dd82196c3aa3648aff]) successfully deleted
W20260806 18:31:25.394120 23620 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.193:45875 (error 108)
I20260806 18:31:25.395237 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 23959
I20260806 18:31:25.408890 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:25.408343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:45875 (local address 127.0.0.1:46140)
0806 18:31:25.408695 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0806 18:31:25.408778 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:45875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252}
W20260806 18:31:25.409315 23620 catalog_manager.cc:4584] TS 0e426170b1ba4c3f84e0e1838dfb18ee (127.7.67.193:45875): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 20820271755d454d9fc55e65becb9c4b: Network error: Client connection negotiation failed: client connection to 127.7.67.193:45875: connect: Connection refused (error 111)
I20260806 18:31:25.410382 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 24155
I20260806 18:31:25.498021 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:25.497574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:45875 (local address 127.0.0.1:46152)
0806 18:31:25.497828 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 18:31:25.497906 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:45875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260806 18:31:25.507798 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:31:25.507452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:45875 (local address 127.0.0.1:46154)
0806 18:31:25.507605 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 18:31:25.507684 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:45875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 18:31:25.512665 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 23591
I20260806 18:31:25.530112 7439 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 18:31:25.876140 7439 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 18:31:25.876271 7439 mini_ranger.cc:66] Stopping Ranger...
I20260806 18:31:26.231151 7439 mini_ranger.cc:68] Stopped Ranger
2026-08-06 18:31:26.231 UTC [22942] LOG: received smart shutdown request
2026-08-06 18:31:26.235 UTC [22942] LOG: background worker "logical replication launcher" (PID 22964) exited with exit code 1
2026-08-06 18:31:26.235 UTC [22950] LOG: shutting down
2026-08-06 18:31:26.378 UTC [22942] LOG: database system is shut down
2026-08-06T18:31:26Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47325 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:31:26 dist-test-slave-5bn2 krb5kdc[24438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:31:26 dist-test-slave-5bn2 krb5kdc[24438](info): set up 2 sockets
Aug 06 18:31:26 dist-test-slave-5bn2 krb5kdc[24438](info): commencing operation
krb5kdc: starting...
W20260806 18:31:27.605537 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:31:27 dist-test-slave-5bn2 krb5kdc[24438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:31:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:31:27Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.67.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.67.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.67.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/rangeradmin_127.7.67.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.67.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/rangeradmin_127.7.67.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.67.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.67.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.67.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/rangerlookup_127.7.67.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.67.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/rangerlookup_127.7.67.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.67.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.67.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.67.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/HTTP_127.7.67.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.67.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/HTTP_127.7.67.212@KRBTEST.COM.keytab.
I20260806 18:31:27.777179 7439 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskM6OdK6/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/postgres -l logfile start
2026-08-06 18:31:30.055 UTC [24481] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 18:31:30.055 UTC [24481] LOG: listening on IPv4 address "127.7.67.212", port 55733
2026-08-06 18:31:30.059 UTC [24481] LOG: listening on Unix socket "/tmp/.s.PGSQL.55733"
2026-08-06 18:31:30.079 UTC [24487] LOG: database system was shut down at 2026-08-06 18:31:28 UTC
2026-08-06 18:31:30.083 UTC [24481] LOG: database system is ready to accept connections
I20260806 18:31:31.104715 7439 mini_postgres.cc:96] Postgres bound to 55733
2026-08-06 18:31:31.112 UTC [24496] FATAL: database "slave" does not exist
127.7.67.212:55733 - accepting connections
I20260806 18:31:31.112670 7439 mini_ranger.cc:161] Starting Ranger...
I20260806 18:31:31.128027 7439 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 18:31:31.550061 7439 mini_ranger.cc:87] Created ranger Postgres database
I20260806 18:31:31.550166 7439 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ranger-admin
2026-08-06 18:31:31,817 [I] DB FLAVOR :POSTGRES
2026-08-06 18:31:31,817 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:31:31,817 [I] Checking connection..
2026-08-06 18:31:31,817 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 18:31:32,062 [I] Checking connection passed.
2026-08-06 18:31:32,062 [I] --------- Verifying version history table ---------
2026-08-06 18:31:32,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:31:32,328 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:31:32,329 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:31:32,582 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:31:32,582 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 18:31:32,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 18:31:32.836 UTC [24582] WARNING: there is no transaction in progress
2026-08-06 18:31:32,844 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 18:31:32,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:31:33,114 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 18:31:33,115 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:31:33,115 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 18:31:33,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 18:31:33,640 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5bn2.c.gcp-upstream.internal','N') ;"
2026-08-06 18:31:33,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:31:34,160 [I] Table x_portal_user does not exist in database ranger
2026-08-06 18:31:34,160 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:31:34,416 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 18:31:34,417 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:31:34,666 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:31:34,921 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 18:31:34,922 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 18:31:35.835 UTC [24741] WARNING: there is no transaction in progress
2026-08-06 18:31:35.851 UTC [24741] WARNING: there is no transaction in progress
2026-08-06 18:31:35.867 UTC [24741] WARNING: there is no transaction in progress
2026-08-06 18:31:35.882 UTC [24741] WARNING: there is no transaction in progress
2026-08-06 18:31:35.901 UTC [24741] WARNING: there is no transaction in progress
2026-08-06 18:31:35.920 UTC [24741] WARNING: there is no transaction in progress
2026-08-06 18:31:35.986 UTC [24741] WARNING: there is no transaction in progress
2026-08-06 18:31:35.992 UTC [24741] WARNING: there is no transaction in progress
2026-08-06 18:31:36.005 UTC [24741] WARNING: there is no transaction in progress
2026-08-06 18:31:36.015 UTC [24741] WARNING: there is no transaction in progress
2026-08-06 18:31:36.024 UTC [24741] WARNING: there is no transaction in progress
2026-08-06 18:31:36.038 UTC [24741] WARNING: there is no transaction in progress
2026-08-06 18:31:36.047 UTC [24741] WARNING: there is no transaction in progress
2026-08-06 18:31:36.053 UTC [24741] WARNING: there is no transaction in progress
2026-08-06 18:31:36.060 UTC [24741] WARNING: there is no transaction in progress
2026-08-06 18:31:36.066 UTC [24741] WARNING: there is no transaction in progress
2026-08-06 18:31:36,643 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 18:31:36,643 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 18:31:36,885 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 18:31:36,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:31:37,135 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 18:31:37,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:31:37,400 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 18:31:37,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:31:37,658 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:31:37,905 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5bn2.c.gcp-upstream.internal';"
2026-08-06 18:31:38,160 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 18:31:38,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 18:31:38,431 [I] DB_PATCHES have already been applied
I20260806 18:31:38.437021 7439 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ranger-admin:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskM6OdK6/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 18:31:38.437088 7439 mini_ranger.cc:193] Using host: 127.7.67.212
I20260806 18:31:38.441267 7439 mini_ranger.cc:227] Ranger admin URL: http://127.7.67.212:37079
Aug 06, 2026 6:31:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:31:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:31:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:31:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:31:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:31:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:31:39 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 18:31:39.614764 7439 mini_ranger.cc:160] Time spent starting Ranger: real 8.502s user 0.003s sys 0.008s
Aug 06, 2026 6:31:59 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:32:00 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 6:32:00 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:32:00 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:32:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:32:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:32:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:32:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:32:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:32:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:32:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:32:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:32:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:32:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:32:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:32:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:32:00 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:32:00 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 18:32:01.562217 7439 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.67.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.67.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.67.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/rangerkms_127.7.67.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.67.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/rangerkms_127.7.67.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.67.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.67.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.67.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/HTTP_127.7.67.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.67.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/HTTP_127.7.67.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 18:32:01 dist-test-slave-5bn2 krb5kdc[24438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041121, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 18:32:01.667403 7439 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 18:32:01.679337 7439 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 18:32:02.429009 7439 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 18:32:02.429090 7439 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ranger-kms
I20260806 18:32:02.429101 7439 mini_ranger_kms.cc:227] Using postgres at 127.7.67.212:55733
2026-08-06 18:32:02,600 [I] DB FLAVOR :POSTGRES
2026-08-06 18:32:02,600 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:32:02,600 [I] Checking connection
2026-08-06 18:32:02,600 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 18:32:02,834 [I] connection success
2026-08-06 18:32:02,835 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 18:32:02,835 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 18:32:02,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 18:32:03,068 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 18:32:03,068 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:32:03,068 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 18:32:03,068 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.212:55733/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 18:32:03,351 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 18:32:03.885895 7439 mini_ranger_kms.cc:310] Created kms service
I20260806 18:32:04.084014 7439 mini_ranger_kms.cc:326] Created kudu user
I20260806 18:32:04.126264 7439 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 18:32:04 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041121, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
I20260806 18:32:04.528842 7439 mini_ranger_kms.cc:384] Added ranger policy
I20260806 18:32:04.528995 7439 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ranger-kms:/tmp/dist-test-taskM6OdK6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskM6OdK6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 18:32:04.529017 7439 mini_ranger_kms.cc:240] Using host: 127.7.67.212
I20260806 18:32:04.534286 7439 mini_ranger_kms.cc:276] Ranger KMS PID: 24993
I20260806 18:32:04.534364 7439 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.67.212:57755
Aug 06, 2026 6:32:05 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:32:05 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:32:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:32:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:32:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:32:06 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 18:32:06 dist-test-slave-5bn2 krb5kdc[24438](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041126, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.67.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:06.995007 7439 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.328s user 0.012s sys 0.143s
I20260806 18:32:06.995111 7439 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 18:32:06.995169 7439 mini_ranger_kms.cc:401] 127.7.67.212:57755/kms/v1/keys
Aug 06 18:32:07 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041126, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.67.212@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
Aug 06 18:32:07 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041126, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.67.212@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
Aug 06 18:32:09 dist-test-slave-5bn2 krb5kdc[24438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:32:09.941821 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:45841
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:45841
--ranger_config_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.212:57755/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:32:10.072345 25072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:10.072630 25072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:10.072670 25072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:10.076725 25072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:32:10.076797 25072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:10.076817 25072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:10.076830 25072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:32:10.076844 25072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:32:10.082605 25072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.67.212:57755/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:45841
--ranger_config_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:45841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:10.083802 25072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:10.084853 25072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:10.093672 25085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:10.093694 25087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:10.093698 25084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:10.093889 25072 server_base.cc:1034] running on GCE node
I20260806 18:32:10.094484 25072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:10.095655 25072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:10.096809 25072 hybrid_clock.cc:648] HybridClock initialized: now 1786041130096797 us; error 26 us; skew 500 ppm
Aug 06 18:32:10 dist-test-slave-5bn2 krb5kdc[24438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041130, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:10.099592 25072 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:32:10.100450 25072 webserver.cc:460] Webserver started at http://127.7.67.254:45131/ using document root <none> and password file <none>
I20260806 18:32:10.100986 25072 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:10.101042 25072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:10.101240 25072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:32:10 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041130, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
I20260806 18:32:10.165097 25072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "9410d1624c814f97b693e40b7b7b4547"
format_stamp: "Formatted at 2026-08-06 18:32:10 on dist-test-slave-5bn2"
server_key: "6decd383fec7f92a9b311161f780b70f"
server_key_iv: "2d9ba2bd9da6465f46796aa0c2a4b42a"
server_key_version: "kuduclusterkey@0"
I20260806 18:32:10.165750 25072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "9410d1624c814f97b693e40b7b7b4547"
format_stamp: "Formatted at 2026-08-06 18:32:10 on dist-test-slave-5bn2"
server_key: "6decd383fec7f92a9b311161f780b70f"
server_key_iv: "2d9ba2bd9da6465f46796aa0c2a4b42a"
server_key_version: "kuduclusterkey@0"
I20260806 18:32:10.169853 25072 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:32:10.188422 25095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:10.189795 25072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:32:10.189932 25072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "9410d1624c814f97b693e40b7b7b4547"
format_stamp: "Formatted at 2026-08-06 18:32:10 on dist-test-slave-5bn2"
server_key: "6decd383fec7f92a9b311161f780b70f"
server_key_iv: "2d9ba2bd9da6465f46796aa0c2a4b42a"
server_key_version: "kuduclusterkey@0"
I20260806 18:32:10.190078 25072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:10.202531 25072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:10.205446 25072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:10.205631 25072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:10.213299 25072 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:45841
I20260806 18:32:10.213310 25170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:45841 every 8 connection(s)
I20260806 18:32:10.214277 25072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:32:10.217051 25171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:32:10.220860 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 25072
I20260806 18:32:10.221009 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:32:10.221324 7439 external_mini_cluster.cc:1428] Setting key 47c6f9a9d4edd300b11b3b4bddaa9d25
I20260806 18:32:10.223395 25171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547: Bootstrap starting.
I20260806 18:32:10.225893 25171 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547: Neither blocks nor log segments found. Creating new log.
I20260806 18:32:10.226797 25171 log.cc:824] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:10.229086 25171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547: No bootstrap required, opened a new log
Aug 06 18:32:10 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:10.233458 25171 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9410d1624c814f97b693e40b7b7b4547" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45841 } }
I20260806 18:32:10.233775 25171 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:32:10.233863 25171 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9410d1624c814f97b693e40b7b7b4547, State: Initialized, Role: FOLLOWER
I20260806 18:32:10.234442 25171 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9410d1624c814f97b693e40b7b7b4547" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45841 } }
I20260806 18:32:10.234623 25171 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:32:10.234743 25171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:32:10.234892 25171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:32:10.236074 25171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9410d1624c814f97b693e40b7b7b4547" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45841 } }
I20260806 18:32:10.236478 25171 leader_election.cc:302] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9410d1624c814f97b693e40b7b7b4547; no voters:
I20260806 18:32:10.236882 25171 leader_election.cc:288] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:32:10.237027 25176 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:32:10.237288 25176 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [term 1 LEADER]: Becoming Leader. State: Replica: 9410d1624c814f97b693e40b7b7b4547, State: Running, Role: LEADER
I20260806 18:32:10.237663 25176 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9410d1624c814f97b693e40b7b7b4547" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45841 } }
I20260806 18:32:10.238394 25171 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:32:10.239094 25176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9410d1624c814f97b693e40b7b7b4547" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9410d1624c814f97b693e40b7b7b4547" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45841 } } }
I20260806 18:32:10.239317 25176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [sys.catalog]: This master's current role is: LEADER
I20260806 18:32:10.239594 25176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9410d1624c814f97b693e40b7b7b4547. Latest consensus state: current_term: 1 leader_uuid: "9410d1624c814f97b693e40b7b7b4547" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9410d1624c814f97b693e40b7b7b4547" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45841 } } }
I20260806 18:32:10.239744 25176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [sys.catalog]: This master's current role is: LEADER
I20260806 18:32:10.240082 25187 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:32:10.242074 25174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:10.222827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32772 (local address 127.7.67.254:45841)
0806 18:32:10.223447 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:10.223476 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:10.223524 (+ 48us) server_negotiation.cc:407] Connection header received
0806 18:32:10.224378 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:10.224403 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:10.224854 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:10.225226 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:10.226249 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:10.227892 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:10.229117 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:10.229867 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:10.230027 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:10.233339 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:10.233346 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:10.233364 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:10.233401 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:10.235601 (+ 2200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:10.236129 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:10.236137 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:10.236144 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:10.236291 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:10.236660 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:10.236663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:10.236665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:10.237247 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:10.238248 (+ 1001us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:10.238417 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 18:32:10.238719 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":158,"threads_started":1}
I20260806 18:32:10.246778 25187 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:32:10.247706 25171 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 18:32:10.257460 25187 catalog_manager.cc:1347] Generated new cluster ID: 69a98c2a530948099eeb34a40a005fee
I20260806 18:32:10.257553 25187 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:32:10.277853 25187 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:32:10.278939 25187 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:32:10.287084 25187 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547: Generated new TSK 0
I20260806 18:32:10.287840 25187 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
I20260806 18:32:10.940243 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.212:57755/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:45841
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:32:11.066269 25234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:11.066562 25234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:11.066624 25234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:11.071069 25234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:11.071141 25234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:11.071221 25234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:32:11.077221 25234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.67.212:57755/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:45841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:11.078399 25234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:11.079589 25234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:11.089649 25244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:11.089643 25241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:11.089645 25242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:11.090128 25234 server_base.cc:1034] running on GCE node
I20260806 18:32:11.090577 25234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:11.091401 25234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:11.092623 25234 hybrid_clock.cc:648] HybridClock initialized: now 1786041131092597 us; error 34 us; skew 500 ppm
Aug 06 18:32:11 dist-test-slave-5bn2 krb5kdc[24438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041131, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:11.096197 25234 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:32:11.097394 25234 webserver.cc:460] Webserver started at http://127.7.67.193:44363/ using document root <none> and password file <none>
I20260806 18:32:11.098150 25234 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:11.098249 25234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:11.098529 25234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:32:11 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041131, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
I20260806 18:32:11.115435 25234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "8985aa9192844a3686933c8fb7119e57"
format_stamp: "Formatted at 2026-08-06 18:32:11 on dist-test-slave-5bn2"
server_key: "bc26164531a1a6a761bd3ac8585bb852"
server_key_iv: "57f4f025172af2562b6c0f4452f61a84"
server_key_version: "kuduclusterkey@0"
I20260806 18:32:11.116255 25234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "8985aa9192844a3686933c8fb7119e57"
format_stamp: "Formatted at 2026-08-06 18:32:11 on dist-test-slave-5bn2"
server_key: "bc26164531a1a6a761bd3ac8585bb852"
server_key_iv: "57f4f025172af2562b6c0f4452f61a84"
server_key_version: "kuduclusterkey@0"
I20260806 18:32:11.122016 25234 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 18:32:11.146189 25254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:11.150055 25234 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.004s sys 0.000s
I20260806 18:32:11.150347 25234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "8985aa9192844a3686933c8fb7119e57"
format_stamp: "Formatted at 2026-08-06 18:32:11 on dist-test-slave-5bn2"
server_key: "bc26164531a1a6a761bd3ac8585bb852"
server_key_iv: "57f4f025172af2562b6c0f4452f61a84"
server_key_version: "kuduclusterkey@0"
I20260806 18:32:11.150602 25234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:11.190649 25234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:11.194542 25234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:11.194818 25234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:11.195672 25234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:32:11.196828 25234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:32:11.196942 25234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:11.197082 25234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:32:11.197171 25234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:11.208267 25234 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:39757
I20260806 18:32:11.209916 25234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:32:11.212085 25413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:39757 every 8 connection(s)
I20260806 18:32:11.216698 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 25234
I20260806 18:32:11.216868 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:32:11.217181 7439 external_mini_cluster.cc:1428] Setting key 960c3c6f1b8b8c8d4b9710e272719278
Aug 06 18:32:11 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041131, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:11.229064 25420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:11.212466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:55833 (local address 127.7.67.254:45841)
0806 18:32:11.214091 (+ 1625us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:11.214100 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:11.215920 (+ 1820us) server_negotiation.cc:407] Connection header received
0806 18:32:11.215969 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:11.215972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:11.216025 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:11.216113 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:11.219059 (+ 2946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:11.219775 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:11.221352 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:11.221692 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:11.221775 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:11.224807 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:11.224811 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:11.224814 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:11.224840 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:11.226665 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:11.227308 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:11.227311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:11.227313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:11.227396 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:11.227750 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:11.227753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:11.227755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:11.227978 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:11.228066 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:11.228736 (+ 670us) server_negotiation.cc:299] Negotiation successful
0806 18:32:11.228869 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1530,"thread_start_us":109,"threads_started":1}
I20260806 18:32:11.230731 25423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:11.212973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:45841 (local address 127.7.67.193:55833)
0806 18:32:11.214541 (+ 1568us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:11.214598 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:11.215622 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:11.217720 (+ 2098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:11.217742 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:11.218134 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:11.218874 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:11.218899 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:11.219930 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:11.219933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:11.221221 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:11.221233 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:11.221952 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:11.221956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:11.222173 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:11.222908 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:11.222930 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:11.224620 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:11.226815 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:11.226821 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:11.226838 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:11.227201 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:11.227488 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:11.227490 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:11.227492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:11.227657 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:11.228133 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:11.228140 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:11.228463 (+ 323us) client_negotiation.cc:769] Sending connection context
0806 18:32:11.229260 (+ 797us) client_negotiation.cc:240] Negotiation successful
0806 18:32:11.229626 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1338,"thread_start_us":153,"threads_started":1}
I20260806 18:32:11.232686 25418 heartbeater.cc:347] Connected to a master server at 127.7.67.254:45841
I20260806 18:32:11.232946 25418 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:11.233538 25418 heartbeater.cc:511] Master 127.7.67.254:45841 requested a full tablet report, sending...
I20260806 18:32:11.235641 25118 ts_manager.cc:194] Registered new tserver with Master: 8985aa9192844a3686933c8fb7119e57 (127.7.67.193:39757)
I20260806 18:32:11.237248 25118 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:55833
WARNING: no policy specified for kudu/127.7.67.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
I20260806 18:32:11.266466 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.212:57755/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:45841
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:32:11 dist-test-slave-5bn2 krb5kdc[24438](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041131, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 18:32:11.434073 25428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:11.434579 25428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:11.434672 25428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:11.440996 25428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:11.441154 25428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:11.441278 25428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:32:11.449397 25428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.67.212:57755/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:45841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:11.450912 25428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:11.452334 25428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:11.465776 25438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:11.465804 25437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:11.466552 25443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:11.467192 25428 server_base.cc:1034] running on GCE node
I20260806 18:32:11.467698 25428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:11.468528 25428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:11.469787 25428 hybrid_clock.cc:648] HybridClock initialized: now 1786041131469714 us; error 90 us; skew 500 ppm
Aug 06 18:32:11 dist-test-slave-5bn2 krb5kdc[24438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041131, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:11.473843 25428 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:32:11.475191 25428 webserver.cc:460] Webserver started at http://127.7.67.194:43447/ using document root <none> and password file <none>
I20260806 18:32:11.475963 25428 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:11.476058 25428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:11.476301 25428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:32:11 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041131, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
I20260806 18:32:11.505975 25428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "3e17c9b996364dda9f11c000df34afca"
format_stamp: "Formatted at 2026-08-06 18:32:11 on dist-test-slave-5bn2"
server_key: "24c63202a389f2d3145983e13ab6efcd"
server_key_iv: "302b33406c54922db0e320367035675a"
server_key_version: "kuduclusterkey@0"
I20260806 18:32:11.507022 25428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "3e17c9b996364dda9f11c000df34afca"
format_stamp: "Formatted at 2026-08-06 18:32:11 on dist-test-slave-5bn2"
server_key: "24c63202a389f2d3145983e13ab6efcd"
server_key_iv: "302b33406c54922db0e320367035675a"
server_key_version: "kuduclusterkey@0"
I20260806 18:32:11.512660 25428 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 18:32:11.534549 25450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:11.536458 25428 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:32:11.536702 25428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "3e17c9b996364dda9f11c000df34afca"
format_stamp: "Formatted at 2026-08-06 18:32:11 on dist-test-slave-5bn2"
server_key: "24c63202a389f2d3145983e13ab6efcd"
server_key_iv: "302b33406c54922db0e320367035675a"
server_key_version: "kuduclusterkey@0"
I20260806 18:32:11.536948 25428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:11.578306 25428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:11.588529 25428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:11.589301 25428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:11.590201 25428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:32:11.591396 25428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:32:11.591502 25428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:11.592479 25428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:32:11.592574 25428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:11.612785 25428 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:45417
I20260806 18:32:11.614092 25428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:32:11.622289 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 25428
I20260806 18:32:11.622439 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:32:11.622745 7439 external_mini_cluster.cc:1428] Setting key 0eec182889a3d8f93e73a9cb109cc5e7
Aug 06 18:32:11 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041131, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:11.615667 25608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:45417 every 8 connection(s)
I20260806 18:32:11.639385 25420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:11.619073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:50219 (local address 127.7.67.254:45841)
0806 18:32:11.619209 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:11.619213 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:11.620231 (+ 1018us) server_negotiation.cc:407] Connection header received
0806 18:32:11.621533 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:11.621537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:11.621598 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:11.621654 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:11.628235 (+ 6581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:11.629080 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:11.630820 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:11.631225 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:11.631300 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:11.635006 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:11.635010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:11.635012 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:11.635042 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:11.636618 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:11.637245 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:11.637248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:11.637250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:11.637339 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:11.637772 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:11.637775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:11.637777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:11.638005 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:11.638085 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:11.639037 (+ 952us) server_negotiation.cc:299] Negotiation successful
0806 18:32:11.639180 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:32:11.641278 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:11.618018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:45841 (local address 127.7.67.194:50219)
0806 18:32:11.620056 (+ 2038us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:11.620115 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:11.621255 (+ 1140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:11.626902 (+ 5647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:11.626914 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:11.627309 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:11.628054 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:11.628076 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:11.629245 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:11.629248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:11.630678 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:11.630691 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:11.631526 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:11.631529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:11.631728 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:11.632997 (+ 1269us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:11.633020 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:11.634775 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:11.636766 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:11.636773 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:11.636792 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:11.637129 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:11.637470 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:11.637473 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:11.637475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:11.637664 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:11.638153 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:11.638160 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:11.638704 (+ 544us) client_negotiation.cc:769] Sending connection context
0806 18:32:11.639839 (+ 1135us) client_negotiation.cc:240] Negotiation successful
0806 18:32:11.640199 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1795,"thread_start_us":141,"threads_started":1}
I20260806 18:32:11.643853 25609 heartbeater.cc:347] Connected to a master server at 127.7.67.254:45841
I20260806 18:32:11.644116 25609 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:11.644716 25609 heartbeater.cc:511] Master 127.7.67.254:45841 requested a full tablet report, sending...
I20260806 18:32:11.645912 25118 ts_manager.cc:194] Registered new tserver with Master: 3e17c9b996364dda9f11c000df34afca (127.7.67.194:45417)
I20260806 18:32:11.646672 25118 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:50219
WARNING: no policy specified for kudu/127.7.67.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
I20260806 18:32:11.692078 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.212:57755/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:45841
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:32:11 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041131, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
W20260806 18:32:11.846196 25618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:11.846468 25618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:11.846531 25618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:11.850732 25618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:11.850800 25618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:11.850908 25618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:32:11.856325 25618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.67.212:57755/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:45841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:11.857478 25618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:11.858551 25618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:11.868297 25628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:11.868481 25630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:11.868597 25627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:11.869495 25618 server_base.cc:1034] running on GCE node
I20260806 18:32:11.869966 25618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:11.870699 25618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:11.871879 25618 hybrid_clock.cc:648] HybridClock initialized: now 1786041131871845 us; error 48 us; skew 500 ppm
Aug 06 18:32:11 dist-test-slave-5bn2 krb5kdc[24438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041131, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:11.874624 25618 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:32:11.875559 25618 webserver.cc:460] Webserver started at http://127.7.67.195:41209/ using document root <none> and password file <none>
I20260806 18:32:11.876144 25618 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:11.876214 25618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:11.876435 25618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:32:11 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041131, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
I20260806 18:32:11.905185 25618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "0b57b4d2cc004cc89a05cef83df994e3"
format_stamp: "Formatted at 2026-08-06 18:32:11 on dist-test-slave-5bn2"
server_key: "b268d498e4a8d3cbc8913e725a414d20"
server_key_iv: "dd971c70fa737a5ceee8f77c47e73f17"
server_key_version: "kuduclusterkey@0"
I20260806 18:32:11.905915 25618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "0b57b4d2cc004cc89a05cef83df994e3"
format_stamp: "Formatted at 2026-08-06 18:32:11 on dist-test-slave-5bn2"
server_key: "b268d498e4a8d3cbc8913e725a414d20"
server_key_iv: "dd971c70fa737a5ceee8f77c47e73f17"
server_key_version: "kuduclusterkey@0"
I20260806 18:32:11.911057 25618 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 18:32:11.929416 25641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:11.931201 25618 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:32:11.931371 25618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "0b57b4d2cc004cc89a05cef83df994e3"
format_stamp: "Formatted at 2026-08-06 18:32:11 on dist-test-slave-5bn2"
server_key: "b268d498e4a8d3cbc8913e725a414d20"
server_key_iv: "dd971c70fa737a5ceee8f77c47e73f17"
server_key_version: "kuduclusterkey@0"
I20260806 18:32:11.931540 25618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:11.949934 25618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:11.953639 25618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:11.953855 25618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:11.954643 25618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:32:11.955874 25618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:32:11.955943 25618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:11.956000 25618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:32:11.956032 25618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:11.967540 25618 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:35021
I20260806 18:32:11.967581 25808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:35021 every 8 connection(s)
I20260806 18:32:11.968840 25618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:32:11.975168 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 25618
I20260806 18:32:11.975297 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:32:11.975577 7439 external_mini_cluster.cc:1428] Setting key 9842feb2ce82f9e1e2bb1458706b670a
Aug 06 18:32:11 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041131, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:11.985252 25420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:11.971050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:43735 (local address 127.7.67.254:45841)
0806 18:32:11.971181 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:11.971186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:11.972370 (+ 1184us) server_negotiation.cc:407] Connection header received
0806 18:32:11.973599 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:11.973603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:11.973666 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:11.973760 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:11.975422 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:11.976279 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:11.977936 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:11.978240 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:11.978322 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:11.981047 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:11.981052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:11.981055 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:11.981083 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:11.982786 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:11.983465 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:11.983469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:11.983471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:11.983595 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:11.984020 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:11.984024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:11.984025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:11.984247 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:11.984319 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:11.984954 (+ 635us) server_negotiation.cc:299] Negotiation successful
0806 18:32:11.985086 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:32:11.986156 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:11.971602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:45841 (local address 127.7.67.195:43735)
0806 18:32:11.972200 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:11.972260 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:11.973317 (+ 1057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:11.974088 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:11.974097 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:11.974479 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:11.975223 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:11.975246 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:11.976451 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:11.976455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:11.977804 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:11.977815 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:11.978533 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:11.978536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:11.978749 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:11.979443 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:11.979460 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:11.980845 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:11.982958 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:11.982964 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:11.982980 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:11.983341 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:11.983710 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:11.983712 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:11.983715 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:11.983906 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:11.984395 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:11.984402 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:11.984690 (+ 288us) client_negotiation.cc:769] Sending connection context
0806 18:32:11.984865 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 18:32:11.985198 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":167,"threads_started":1}
I20260806 18:32:11.987309 25809 heartbeater.cc:347] Connected to a master server at 127.7.67.254:45841
I20260806 18:32:11.987567 25809 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:11.988080 25809 heartbeater.cc:511] Master 127.7.67.254:45841 requested a full tablet report, sending...
I20260806 18:32:11.989315 25118 ts_manager.cc:194] Registered new tserver with Master: 0b57b4d2cc004cc89a05cef83df994e3 (127.7.67.195:35021)
I20260806 18:32:11.989971 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:32:11.990046 25118 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:43735
I20260806 18:32:11.990835 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 25234
I20260806 18:32:12.031347 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 25428
I20260806 18:32:12.043861 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 25618
I20260806 18:32:12.058692 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 25072
I20260806 18:32:12.071344 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:45841
--webserver_interface=127.7.67.254
--webserver_port=45131
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:45841
--ranger_config_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.212:57755/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.67.212:57755/kms with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:32:12.182809 25818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:12.183117 25818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:12.183156 25818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:12.187341 25818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:32:12.187407 25818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:12.187425 25818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:12.187443 25818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:32:12.187479 25818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:32:12.192978 25818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.67.212:57755/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:45841
--ranger_config_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:45841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=45131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:12.194046 25818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:12.195112 25818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:12.203570 25829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:12.203570 25826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:12.203591 25827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:12.203724 25818 server_base.cc:1034] running on GCE node
I20260806 18:32:12.204414 25818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:12.205511 25818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:12.206668 25818 hybrid_clock.cc:648] HybridClock initialized: now 1786041132206647 us; error 34 us; skew 500 ppm
Aug 06 18:32:12 dist-test-slave-5bn2 krb5kdc[24438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041132, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:12.209441 25818 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:32:12.210369 25818 webserver.cc:460] Webserver started at http://127.7.67.254:45131/ using document root <none> and password file <none>
I20260806 18:32:12.210984 25818 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:12.211071 25818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:32:12 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041132, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
I20260806 18:32:12.256652 25818 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:32:12.259156 25838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:12.260497 25818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:32:12.260637 25818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "9410d1624c814f97b693e40b7b7b4547"
format_stamp: "Formatted at 2026-08-06 18:32:10 on dist-test-slave-5bn2"
server_key: "6decd383fec7f92a9b311161f780b70f"
server_key_iv: "2d9ba2bd9da6465f46796aa0c2a4b42a"
server_key_version: "kuduclusterkey@0"
I20260806 18:32:12.261214 25818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:12.309227 25818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:12.311964 25818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:12.312156 25818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:12.319222 25818 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:45841
I20260806 18:32:12.319247 25916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:45841 every 8 connection(s)
I20260806 18:32:12.320325 25818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:32:12.324767 25918 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:32:12.326545 25918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547: Bootstrap starting.
I20260806 18:32:12.329262 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 25818
I20260806 18:32:12.329715 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.212:57755/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:39757
--local_ip_for_outbound_sockets=127.7.67.193
--tserver_master_addrs=127.7.67.254:45841
--webserver_port=44363
--webserver_interface=127.7.67.193
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.67.212:57755/kms with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:32:12.343217 25918 log.cc:824] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:12.350299 25918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:32:12.350701 25918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547: Bootstrap complete.
I20260806 18:32:12.354729 25918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9410d1624c814f97b693e40b7b7b4547" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45841 } }
I20260806 18:32:12.355193 25918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9410d1624c814f97b693e40b7b7b4547, State: Initialized, Role: FOLLOWER
I20260806 18:32:12.355677 25918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9410d1624c814f97b693e40b7b7b4547" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45841 } }
I20260806 18:32:12.355799 25918 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:32:12.355865 25918 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:32:12.355959 25918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:32:12.357892 25918 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9410d1624c814f97b693e40b7b7b4547" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45841 } }
I20260806 18:32:12.358212 25918 leader_election.cc:302] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9410d1624c814f97b693e40b7b7b4547; no voters:
I20260806 18:32:12.358521 25918 leader_election.cc:288] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:32:12.358628 25923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:32:12.358896 25923 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [term 2 LEADER]: Becoming Leader. State: Replica: 9410d1624c814f97b693e40b7b7b4547, State: Running, Role: LEADER
I20260806 18:32:12.359270 25923 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9410d1624c814f97b693e40b7b7b4547" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45841 } }
I20260806 18:32:12.359843 25918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:32:12.360809 25926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9410d1624c814f97b693e40b7b7b4547. Latest consensus state: current_term: 2 leader_uuid: "9410d1624c814f97b693e40b7b7b4547" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9410d1624c814f97b693e40b7b7b4547" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45841 } } }
I20260806 18:32:12.360833 25924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9410d1624c814f97b693e40b7b7b4547" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9410d1624c814f97b693e40b7b7b4547" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45841 } } }
I20260806 18:32:12.360944 25924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [sys.catalog]: This master's current role is: LEADER
I20260806 18:32:12.360944 25926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547 [sys.catalog]: This master's current role is: LEADER
I20260806 18:32:12.362135 25918 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 18:32:12.363176 25936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:32:12.369181 25936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:32:12.369944 25936 catalog_manager.cc:1259] Loaded cluster ID: 69a98c2a530948099eeb34a40a005fee
I20260806 18:32:12.370052 25936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:32:12.372694 25936 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:32:12.373438 25936 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547: Loaded TSK: 0
I20260806 18:32:12.374747 25936 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:32:12.473029 25920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:12.473532 25920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:12.473685 25920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:12.478703 25920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:12.478770 25920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:12.478871 25920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:32:12.484405 25920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.67.212:57755/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:39757
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=44363
--tserver_master_addrs=127.7.67.254:45841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:12.485525 25920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:12.486676 25920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:12.496840 25968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:12.497095 25920 server_base.cc:1034] running on GCE node
W20260806 18:32:12.496872 25970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:12.497411 25964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:12.497925 25920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:12.498611 25920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:12.501920 25920 hybrid_clock.cc:648] HybridClock initialized: now 1786041132501659 us; error 262 us; skew 500 ppm
Aug 06 18:32:12 dist-test-slave-5bn2 krb5kdc[24438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041132, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:12.505270 25920 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:32:12.506490 25920 webserver.cc:460] Webserver started at http://127.7.67.193:44363/ using document root <none> and password file <none>
I20260806 18:32:12.507305 25920 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:12.507453 25920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:32:12 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041132, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
I20260806 18:32:12.533901 25920 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:32:12.538012 25982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:12.539547 25920 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260806 18:32:12.539680 25920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "8985aa9192844a3686933c8fb7119e57"
format_stamp: "Formatted at 2026-08-06 18:32:11 on dist-test-slave-5bn2"
server_key: "bc26164531a1a6a761bd3ac8585bb852"
server_key_iv: "57f4f025172af2562b6c0f4452f61a84"
server_key_version: "kuduclusterkey@0"
I20260806 18:32:12.540370 25920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:12.557284 25920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:12.561054 25920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:12.561342 25920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:12.562182 25920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:32:12.563390 25920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:32:12.563519 25920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:12.563659 25920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:32:12.564670 25920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 18:32:12.576663 25920 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:39757
I20260806 18:32:12.577858 25920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:32:12.576692 26143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:39757 every 8 connection(s)
I20260806 18:32:12.588267 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 25920
I20260806 18:32:12.588836 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.212:57755/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:45417
--local_ip_for_outbound_sockets=127.7.67.194
--tserver_master_addrs=127.7.67.254:45841
--webserver_port=43447
--webserver_interface=127.7.67.194
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.67.212:57755/kms with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:32:12 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041132, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:12.606700 26149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:12.581259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:45841 (local address 127.7.67.193:57161)
0806 18:32:12.582045 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:12.582100 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:12.583040 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:12.585057 (+ 2017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:12.585067 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:12.585469 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:12.586183 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:12.586206 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:12.588221 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:12.588225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:12.589689 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:12.589703 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:12.591130 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:12.591135 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:12.591327 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:12.592007 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:12.592031 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:12.593843 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:12.602295 (+ 8452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:12.602304 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:12.602325 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:12.602781 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:12.603331 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:12.603335 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:12.603337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:12.603569 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:12.604514 (+ 945us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:12.604521 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:12.604857 (+ 336us) client_negotiation.cc:769] Sending connection context
0806 18:32:12.605068 (+ 211us) client_negotiation.cc:240] Negotiation successful
0806 18:32:12.605480 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":143,"threads_started":1}
I20260806 18:32:12.608969 26147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:12.581111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:57161 (local address 127.7.67.254:45841)
0806 18:32:12.581755 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:12.581779 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:12.582226 (+ 447us) server_negotiation.cc:407] Connection header received
0806 18:32:12.583677 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:12.583692 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:12.584139 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:12.584525 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:12.586398 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:12.588035 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:12.589849 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:12.590644 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:12.590833 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:12.594107 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:12.594115 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:12.594135 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:12.594171 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:12.602093 (+ 7922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:12.602974 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:12.602983 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:12.602990 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:12.603197 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:12.603695 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:12.603699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:12.603701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:12.604296 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:12.604449 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:12.607525 (+ 3076us) server_negotiation.cc:299] Negotiation successful
0806 18:32:12.607894 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":124,"threads_started":1}
I20260806 18:32:12.612841 26144 heartbeater.cc:347] Connected to a master server at 127.7.67.254:45841
I20260806 18:32:12.613138 26144 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:12.613796 26144 heartbeater.cc:511] Master 127.7.67.254:45841 requested a full tablet report, sending...
W20260806 18:32:12.616392 26144 heartbeater.cc:644] Failed to heartbeat to 127.7.67.254:45841 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 18:32:12.616549 26144 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:12.616760 26144 heartbeater.cc:511] Master 127.7.67.254:45841 requested a full tablet report, sending...
I20260806 18:32:12.617826 26144 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:12.618016 26144 heartbeater.cc:511] Master 127.7.67.254:45841 requested a full tablet report, sending...
W20260806 18:32:12.618882 26144 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:32:12.753295 26150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:12.753787 26150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:12.753921 26150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:12.759251 26150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:12.759402 26150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:12.759552 26150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:32:12.766530 26150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.67.212:57755/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:45417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=43447
--tserver_master_addrs=127.7.67.254:45841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:12.767735 26150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:12.768802 26150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:12.778795 26156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:12.779018 26155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:12.778728 26160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:12.779156 26150 server_base.cc:1034] running on GCE node
I20260806 18:32:12.779620 26150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:12.780272 26150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:12.781435 26150 hybrid_clock.cc:648] HybridClock initialized: now 1786041132781403 us; error 41 us; skew 500 ppm
Aug 06 18:32:12 dist-test-slave-5bn2 krb5kdc[24438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041132, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:12.784739 26150 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:32:12.786096 26150 webserver.cc:460] Webserver started at http://127.7.67.194:43447/ using document root <none> and password file <none>
I20260806 18:32:12.786891 26150 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:12.787067 26150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:32:12 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041132, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
I20260806 18:32:12.824685 26150 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.003s sys 0.000s
I20260806 18:32:12.835673 26173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:12.837466 26150 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.003s sys 0.000s
I20260806 18:32:12.837625 26150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "3e17c9b996364dda9f11c000df34afca"
format_stamp: "Formatted at 2026-08-06 18:32:11 on dist-test-slave-5bn2"
server_key: "24c63202a389f2d3145983e13ab6efcd"
server_key_iv: "302b33406c54922db0e320367035675a"
server_key_version: "kuduclusterkey@0"
I20260806 18:32:12.838454 26150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:12.865651 26150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:12.869575 26150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:12.869866 26150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:12.870692 26150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:32:12.871908 26150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:32:12.872009 26150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:12.872098 26150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:32:12.872149 26150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:12.882906 26150 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:45417
I20260806 18:32:12.884359 26150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:32:12.887095 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 26150
I20260806 18:32:12.887598 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.212:57755/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:35021
--local_ip_for_outbound_sockets=127.7.67.195
--tserver_master_addrs=127.7.67.254:45841
--webserver_port=41209
--webserver_interface=127.7.67.195
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.67.212:57755/kms with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:32:12.892460 26333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:45417 every 8 connection(s)
Aug 06 18:32:12 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041132, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:12.930521 26147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:12.897285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:42543 (local address 127.7.67.254:45841)
0806 18:32:12.897431 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:12.897436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:12.910937 (+ 13501us) server_negotiation.cc:407] Connection header received
0806 18:32:12.912806 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:12.912810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:12.912876 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:12.912927 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:12.915094 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:12.915882 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:12.917661 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:12.918018 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:12.918099 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:12.924191 (+ 6092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:12.924195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:12.924198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:12.924228 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:12.926205 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:12.926889 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:12.926893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:12.926895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:12.926999 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:12.927612 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:12.927616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:12.927618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:12.927847 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:12.927922 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:12.930168 (+ 2246us) server_negotiation.cc:299] Negotiation successful
0806 18:32:12.930330 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:32:12.932291 26338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:12.897921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:45841 (local address 127.7.67.194:42543)
0806 18:32:12.910631 (+ 12710us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:12.910752 (+ 121us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:12.912520 (+ 1768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:12.913409 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:12.913419 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:12.914105 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:12.914890 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:12.914915 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:12.916077 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:12.916080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:12.917508 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:12.917520 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:12.918346 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:12.918351 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:12.918553 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:12.921628 (+ 3075us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:12.921652 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:12.923956 (+ 2304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:12.926342 (+ 2386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:12.926349 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:12.926368 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:12.926739 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:12.927305 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:12.927309 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:12.927311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:12.927499 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:12.927997 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:12.928003 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:12.928325 (+ 322us) client_negotiation.cc:769] Sending connection context
0806 18:32:12.930732 (+ 2407us) client_negotiation.cc:240] Negotiation successful
0806 18:32:12.931165 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12253,"thread_start_us":153,"threads_started":1}
I20260806 18:32:12.933964 26334 heartbeater.cc:347] Connected to a master server at 127.7.67.254:45841
I20260806 18:32:12.934226 26334 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:12.935014 26334 heartbeater.cc:511] Master 127.7.67.254:45841 requested a full tablet report, sending...
W20260806 18:32:12.936899 26334 heartbeater.cc:644] Failed to heartbeat to 127.7.67.254:45841 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 18:32:12.937021 26334 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:12.937184 26334 heartbeater.cc:511] Master 127.7.67.254:45841 requested a full tablet report, sending...
I20260806 18:32:12.938099 26334 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:12.938273 26334 heartbeater.cc:511] Master 127.7.67.254:45841 requested a full tablet report, sending...
W20260806 18:32:12.939152 26334 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:32:13.048379 26337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:13.048657 26337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:13.048699 26337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:13.054622 26337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:13.054780 26337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:13.054947 26337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:32:13.063256 26337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.67.212:57755/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:35021
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=41209
--tserver_master_addrs=127.7.67.254:45841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:13.064706 26337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:13.066180 26337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:13.079633 26347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:13.081267 26337 server_base.cc:1034] running on GCE node
W20260806 18:32:13.082991 26353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:13.084846 26350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:13.085554 26337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:13.086442 26337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:13.089741 26337 hybrid_clock.cc:648] HybridClock initialized: now 1786041133089705 us; error 51 us; skew 500 ppm
Aug 06 18:32:13 dist-test-slave-5bn2 krb5kdc[24438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041133, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:13.092783 26337 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:32:13.094065 26337 webserver.cc:460] Webserver started at http://127.7.67.195:41209/ using document root <none> and password file <none>
I20260806 18:32:13.094645 26337 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:13.094729 26337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:32:13 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041133, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
I20260806 18:32:13.112677 26337 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:32:13.116305 26363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:13.118109 26337 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260806 18:32:13.118251 26337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "0b57b4d2cc004cc89a05cef83df994e3"
format_stamp: "Formatted at 2026-08-06 18:32:11 on dist-test-slave-5bn2"
server_key: "b268d498e4a8d3cbc8913e725a414d20"
server_key_iv: "dd971c70fa737a5ceee8f77c47e73f17"
server_key_version: "kuduclusterkey@0"
I20260806 18:32:13.119074 26337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:13.145463 26337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:13.149571 26337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:13.149868 26337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:13.150763 26337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:32:13.151983 26337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:32:13.152112 26337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:13.152228 26337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:32:13.152297 26337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:13.163439 26337 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:35021
I20260806 18:32:13.163790 26517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:35021 every 8 connection(s)
I20260806 18:32:13.164676 26337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:32:13.174674 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 26337
Aug 06 18:32:13 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041133, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:13.180214 26147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:13.166705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:44741 (local address 127.7.67.254:45841)
0806 18:32:13.166886 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:13.166892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:13.167841 (+ 949us) server_negotiation.cc:407] Connection header received
0806 18:32:13.168800 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:13.168804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:13.168876 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:13.168990 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:13.170050 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:13.170889 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:13.172624 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:13.172845 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:13.172924 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:13.175967 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:13.175971 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:13.175974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:13.176001 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:13.177604 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:13.178205 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:13.178209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:13.178211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:13.178312 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:13.178766 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:13.178770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:13.178772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:13.179038 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:13.179119 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:13.179902 (+ 783us) server_negotiation.cc:299] Negotiation successful
0806 18:32:13.180044 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:32:13.182024 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:13.167181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:45841 (local address 127.7.67.195:44741)
0806 18:32:13.167662 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:13.167716 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:13.168582 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:13.169137 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:13.169144 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:13.169410 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:13.169893 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:13.169907 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:13.171052 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:13.171056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:13.172476 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:13.172489 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:13.173012 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:13.173018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:13.173232 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:13.173881 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:13.173903 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:13.175798 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:13.177719 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:13.177726 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:13.177745 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:13.178098 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:13.178426 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:13.178430 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:13.178432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:13.178646 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:13.179212 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:13.179218 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:13.179587 (+ 369us) client_negotiation.cc:769] Sending connection context
0806 18:32:13.179837 (+ 250us) client_negotiation.cc:240] Negotiation successful
0806 18:32:13.180222 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":108,"threads_started":1}
I20260806 18:32:13.184182 26518 heartbeater.cc:347] Connected to a master server at 127.7.67.254:45841
I20260806 18:32:13.184455 26518 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:13.185109 26518 heartbeater.cc:511] Master 127.7.67.254:45841 requested a full tablet report, sending...
W20260806 18:32:13.187417 26518 heartbeater.cc:644] Failed to heartbeat to 127.7.67.254:45841 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 18:32:13.187572 26518 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:13.187772 26518 heartbeater.cc:511] Master 127.7.67.254:45841 requested a full tablet report, sending...
I20260806 18:32:13.188716 26518 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:13.188895 26518 heartbeater.cc:511] Master 127.7.67.254:45841 requested a full tablet report, sending...
I20260806 18:32:13.189040 26525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:13.175500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32784 (local address 127.7.67.254:45841)
0806 18:32:13.175781 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:13.175786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:13.176077 (+ 291us) server_negotiation.cc:407] Connection header received
0806 18:32:13.176208 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:13.176211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:13.176262 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:13.176346 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:13.177233 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:13.177999 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:13.180419 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:13.180748 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:13.180856 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:13.182813 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:13.182817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:13.182820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:13.182870 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:13.184518 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:13.186463 (+ 1945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:13.186466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:13.186468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:13.186558 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:13.186903 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:13.186911 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:13.186912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:13.187141 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:13.187345 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:13.188716 (+ 1371us) server_negotiation.cc:299] Negotiation successful
0806 18:32:13.188854 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":101,"threads_started":1}
W20260806 18:32:13.190239 26518 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 18:32:13 dist-test-slave-5bn2 krb5kdc[24438](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041133, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:13.619119 26144 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:13.619434 26144 heartbeater.cc:511] Master 127.7.67.254:45841 requested a full tablet report, sending...
I20260806 18:32:13.620807 25866 ts_manager.cc:194] Registered new tserver with Master: 8985aa9192844a3686933c8fb7119e57 (127.7.67.193:39757)
I20260806 18:32:13.622596 25866 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:57161
Aug 06 18:32:13 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041133, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
I20260806 18:32:13.939399 26334 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:13.939725 26334 heartbeater.cc:511] Master 127.7.67.254:45841 requested a full tablet report, sending...
I20260806 18:32:13.940522 25865 ts_manager.cc:194] Registered new tserver with Master: 3e17c9b996364dda9f11c000df34afca (127.7.67.194:45417)
I20260806 18:32:13.941354 25865 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:42543
I20260806 18:32:14.190475 26518 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:14.190819 26518 heartbeater.cc:511] Master 127.7.67.254:45841 requested a full tablet report, sending...
I20260806 18:32:14.192015 25865 ts_manager.cc:194] Registered new tserver with Master: 0b57b4d2cc004cc89a05cef83df994e3 (127.7.67.195:35021)
I20260806 18:32:14.192772 25865 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:44741
I20260806 18:32:14.198611 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:32:14.215482 26549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:14.200308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32790 (local address 127.7.67.254:45841)
0806 18:32:14.200995 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:14.200999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:14.201310 (+ 311us) server_negotiation.cc:407] Connection header received
0806 18:32:14.201351 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:14.201354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:14.201400 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:14.201468 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:14.202344 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.203358 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.204497 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.204844 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.204931 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:14.211263 (+ 6332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:14.211268 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:14.211270 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:14.211302 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:14.213138 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:14.213650 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:14.213653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:14.213655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:14.213750 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:14.214039 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:14.214043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:14.214045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:14.214278 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:14.214399 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:14.215157 (+ 758us) server_negotiation.cc:299] Negotiation successful
0806 18:32:14.215290 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":107,"threads_started":1}
I20260806 18:32:14.220333 25865 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:32:14.226246 25865 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 06 18:32:14 dist-test-slave-5bn2 krb5kdc[24438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041133, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for HTTP/127.7.67.212@KRBTEST.COM
I20260806 18:32:14.279846 26555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:14.263991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:39757 (local address 127.0.0.1:51696)
0806 18:32:14.264816 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:14.265132 (+ 316us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:14.265273 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:14.266001 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:14.266007 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:14.266042 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:14.266194 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:14.266200 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.276238 (+ 10038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.276242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:14.277820 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:14.277834 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.279228 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.279232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:14.279347 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:14.279391 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 18:32:14.279503 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 18:32:14.279661 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"mutex_wait_us":14,"thread_start_us":114,"threads_started":1}
I20260806 18:32:14.280378 26558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:14.265784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:45417 (local address 127.0.0.1:47916)
0806 18:32:14.271339 (+ 5555us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:14.271354 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:14.271460 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:14.272149 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:14.272153 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:14.272174 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:14.272826 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:14.272839 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.276238 (+ 3399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.276242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:14.277820 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:14.277834 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.279833 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.279837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:14.279918 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:14.279941 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:32:14.280167 (+ 226us) client_negotiation.cc:240] Negotiation successful
0806 18:32:14.280249 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5490,"mutex_wait_us":330,"thread_start_us":101,"threads_started":1}
I20260806 18:32:14.280587 26556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:14.264149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51696 (local address 127.7.67.193:39757)
0806 18:32:14.265051 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:14.265059 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:14.265418 (+ 359us) server_negotiation.cc:407] Connection header received
0806 18:32:14.265502 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:14.265558 (+ 56us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:14.265818 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:14.266341 (+ 523us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:14.266488 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.276092 (+ 9604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.277983 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.278874 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.278983 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:14.279155 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:14.280341 (+ 1186us) server_negotiation.cc:299] Negotiation successful
0806 18:32:14.280452 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":132,"threads_started":1}
I20260806 18:32:14.284591 26036 tablet_service.cc:1467] Processing CreateTablet for tablet 8d358055e3a54532b929d7365ee71cc9 (DEFAULT_TABLE table=test-table [id=6d654d22e2874492ae4dc6c56736244f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:32:14.287109 26036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d358055e3a54532b929d7365ee71cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:32:14.287479 26557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:14.265463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:35021 (local address 127.0.0.1:48272)
0806 18:32:14.271041 (+ 5578us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:14.271059 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:14.271159 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:14.273106 (+ 1947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:14.273109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:14.273125 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:14.273207 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:14.273214 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.284369 (+ 11155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.284373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:14.285570 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:14.285586 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.287009 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.287013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:14.287125 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:14.287150 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:32:14.287222 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 18:32:14.287300 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5493,"thread_start_us":87,"threads_started":1}
I20260806 18:32:14.288522 26560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:14.266219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48272 (local address 127.7.67.195:35021)
0806 18:32:14.272525 (+ 6306us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:14.272531 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:14.272551 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:32:14.272615 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:14.272619 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:14.272784 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:14.272937 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:14.282953 (+ 10016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.284200 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.285744 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.286636 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.286746 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:14.286929 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:14.288250 (+ 1321us) server_negotiation.cc:299] Negotiation successful
0806 18:32:14.288363 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6197,"thread_start_us":94,"threads_started":1}
I20260806 18:32:14.291976 26431 tablet_service.cc:1467] Processing CreateTablet for tablet 8d358055e3a54532b929d7365ee71cc9 (DEFAULT_TABLE table=test-table [id=6d654d22e2874492ae4dc6c56736244f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:32:14.293191 26431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d358055e3a54532b929d7365ee71cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:32:14.295930 26559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:14.265968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47916 (local address 127.7.67.194:45417)
0806 18:32:14.271704 (+ 5736us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:14.271710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:14.271732 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:32:14.271805 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:14.271809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:14.272003 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:14.272178 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:14.272969 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.276098 (+ 3129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.277977 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.278879 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.278983 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:14.295315 (+ 16332us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:14.295432 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 18:32:14.295567 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5625,"thread_start_us":86,"threads_started":1}
I20260806 18:32:14.300853 26235 tablet_service.cc:1467] Processing CreateTablet for tablet 8d358055e3a54532b929d7365ee71cc9 (DEFAULT_TABLE table=test-table [id=6d654d22e2874492ae4dc6c56736244f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:32:14.302057 26235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d358055e3a54532b929d7365ee71cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:32:14.303627 26564 tablet_bootstrap.cc:492] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3: Bootstrap starting.
I20260806 18:32:14.309346 26564 tablet_bootstrap.cc:654] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3: Neither blocks nor log segments found. Creating new log.
I20260806 18:32:14.310297 26564 log.cc:824] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:14.311049 26566 tablet_bootstrap.cc:492] T 8d358055e3a54532b929d7365ee71cc9 P 3e17c9b996364dda9f11c000df34afca: Bootstrap starting.
I20260806 18:32:14.312024 26563 tablet_bootstrap.cc:492] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57: Bootstrap starting.
I20260806 18:32:14.314553 26564 tablet_bootstrap.cc:492] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3: No bootstrap required, opened a new log
I20260806 18:32:14.314828 26564 ts_tablet_manager.cc:1397] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260806 18:32:14.319262 26564 raft_consensus.cc:348] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8985aa9192844a3686933c8fb7119e57" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39757 } } peers { permanent_uuid: "0b57b4d2cc004cc89a05cef83df994e3" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35021 } } peers { permanent_uuid: "3e17c9b996364dda9f11c000df34afca" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45417 } }
I20260806 18:32:14.321017 26564 raft_consensus.cc:374] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:32:14.321149 26564 raft_consensus.cc:729] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b57b4d2cc004cc89a05cef83df994e3, State: Initialized, Role: FOLLOWER
I20260806 18:32:14.321434 26563 tablet_bootstrap.cc:654] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57: Neither blocks nor log segments found. Creating new log.
I20260806 18:32:14.321779 26564 consensus_queue.cc:260] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8985aa9192844a3686933c8fb7119e57" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39757 } } peers { permanent_uuid: "0b57b4d2cc004cc89a05cef83df994e3" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35021 } } peers { permanent_uuid: "3e17c9b996364dda9f11c000df34afca" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45417 } }
I20260806 18:32:14.322830 26563 log.cc:824] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:14.323038 26564 ts_tablet_manager.cc:1428] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20260806 18:32:14.323431 26518 heartbeater.cc:503] Master 127.7.67.254:45841 was elected leader, sending a full tablet report...
I20260806 18:32:14.339790 26566 tablet_bootstrap.cc:654] T 8d358055e3a54532b929d7365ee71cc9 P 3e17c9b996364dda9f11c000df34afca: Neither blocks nor log segments found. Creating new log.
I20260806 18:32:14.340610 26566 log.cc:824] T 8d358055e3a54532b929d7365ee71cc9 P 3e17c9b996364dda9f11c000df34afca: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:14.343302 26563 tablet_bootstrap.cc:492] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57: No bootstrap required, opened a new log
I20260806 18:32:14.343542 26563 ts_tablet_manager.cc:1397] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.003s
I20260806 18:32:14.347347 26563 raft_consensus.cc:348] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8985aa9192844a3686933c8fb7119e57" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39757 } } peers { permanent_uuid: "0b57b4d2cc004cc89a05cef83df994e3" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35021 } } peers { permanent_uuid: "3e17c9b996364dda9f11c000df34afca" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45417 } }
I20260806 18:32:14.349413 26566 tablet_bootstrap.cc:492] T 8d358055e3a54532b929d7365ee71cc9 P 3e17c9b996364dda9f11c000df34afca: No bootstrap required, opened a new log
I20260806 18:32:14.349531 26563 raft_consensus.cc:374] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:32:14.349610 26563 raft_consensus.cc:729] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8985aa9192844a3686933c8fb7119e57, State: Initialized, Role: FOLLOWER
I20260806 18:32:14.350128 26563 consensus_queue.cc:260] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8985aa9192844a3686933c8fb7119e57" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39757 } } peers { permanent_uuid: "0b57b4d2cc004cc89a05cef83df994e3" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35021 } } peers { permanent_uuid: "3e17c9b996364dda9f11c000df34afca" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45417 } }
I20260806 18:32:14.350814 26144 heartbeater.cc:503] Master 127.7.67.254:45841 was elected leader, sending a full tablet report...
I20260806 18:32:14.351256 26563 ts_tablet_manager.cc:1428] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20260806 18:32:14.351513 26566 ts_tablet_manager.cc:1397] T 8d358055e3a54532b929d7365ee71cc9 P 3e17c9b996364dda9f11c000df34afca: Time spent bootstrapping tablet: real 0.041s user 0.006s sys 0.000s
I20260806 18:32:14.355077 26566 raft_consensus.cc:348] T 8d358055e3a54532b929d7365ee71cc9 P 3e17c9b996364dda9f11c000df34afca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8985aa9192844a3686933c8fb7119e57" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39757 } } peers { permanent_uuid: "0b57b4d2cc004cc89a05cef83df994e3" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35021 } } peers { permanent_uuid: "3e17c9b996364dda9f11c000df34afca" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45417 } }
I20260806 18:32:14.355307 26566 raft_consensus.cc:374] T 8d358055e3a54532b929d7365ee71cc9 P 3e17c9b996364dda9f11c000df34afca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:32:14.355568 26566 raft_consensus.cc:729] T 8d358055e3a54532b929d7365ee71cc9 P 3e17c9b996364dda9f11c000df34afca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e17c9b996364dda9f11c000df34afca, State: Initialized, Role: FOLLOWER
I20260806 18:32:14.356091 26566 consensus_queue.cc:260] T 8d358055e3a54532b929d7365ee71cc9 P 3e17c9b996364dda9f11c000df34afca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8985aa9192844a3686933c8fb7119e57" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39757 } } peers { permanent_uuid: "0b57b4d2cc004cc89a05cef83df994e3" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35021 } } peers { permanent_uuid: "3e17c9b996364dda9f11c000df34afca" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45417 } }
I20260806 18:32:14.357246 26566 ts_tablet_manager.cc:1428] T 8d358055e3a54532b929d7365ee71cc9 P 3e17c9b996364dda9f11c000df34afca: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:32:14.357632 26334 heartbeater.cc:503] Master 127.7.67.254:45841 was elected leader, sending a full tablet report...
W20260806 18:32:14.397136 26335 tablet.cc:2369] T 8d358055e3a54532b929d7365ee71cc9 P 3e17c9b996364dda9f11c000df34afca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:32:14.417498 26520 tablet.cc:2369] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:32:14.580673 26145 tablet.cc:2369] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:32:14.623272 26571 raft_consensus.cc:482] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:32:14.623557 26571 raft_consensus.cc:504] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8985aa9192844a3686933c8fb7119e57" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39757 } } peers { permanent_uuid: "0b57b4d2cc004cc89a05cef83df994e3" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35021 } } peers { permanent_uuid: "3e17c9b996364dda9f11c000df34afca" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45417 } }
I20260806 18:32:14.624856 26571 leader_election.cc:288] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b57b4d2cc004cc89a05cef83df994e3 (127.7.67.195:35021), 3e17c9b996364dda9f11c000df34afca (127.7.67.194:45417)
I20260806 18:32:14.629381 26560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:14.625216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:40735 (local address 127.7.67.195:35021)
0806 18:32:14.625363 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:14.625369 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:14.625558 (+ 189us) server_negotiation.cc:407] Connection header received
0806 18:32:14.625792 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:14.625796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:14.625853 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:14.625904 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:14.626303 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.627092 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.628214 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.628684 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.628795 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:14.628850 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:14.629161 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 18:32:14.629245 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:32:14.629899 26587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:14.625147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:35021 (local address 127.7.67.193:40735)
0806 18:32:14.625459 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:14.625472 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:14.625652 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:14.626038 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:14.626041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:14.626057 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:14.626191 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:14.626198 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.627209 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.627213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:14.628080 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:14.628090 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.628922 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.628925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:14.629020 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:14.629045 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:32:14.629684 (+ 639us) client_negotiation.cc:240] Negotiation successful
0806 18:32:14.629767 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":122,"threads_started":1}
I20260806 18:32:14.629942 26559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:14.625719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:49639 (local address 127.7.67.194:45417)
0806 18:32:14.625842 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:14.625847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:14.626128 (+ 281us) server_negotiation.cc:407] Connection header received
0806 18:32:14.626236 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:14.626240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:14.626294 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:14.626362 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:14.626619 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.627156 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.628641 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.629247 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.629342 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:14.629391 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:14.629725 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 18:32:14.629806 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:32:14.630337 26588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:14.625566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:45417 (local address 127.7.67.193:49639)
0806 18:32:14.626007 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:14.626020 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:14.626148 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:14.626392 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:14.626394 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:14.626410 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:14.626514 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:14.626519 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.627259 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.627263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:14.628493 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:14.628505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.629485 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.629489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:14.629594 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:14.629615 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:32:14.630117 (+ 502us) client_negotiation.cc:240] Negotiation successful
0806 18:32:14.630193 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":101,"threads_started":1}
I20260806 18:32:14.630607 26459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d358055e3a54532b929d7365ee71cc9" candidate_uuid: "8985aa9192844a3686933c8fb7119e57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b57b4d2cc004cc89a05cef83df994e3" is_pre_election: true
I20260806 18:32:14.630996 26459 raft_consensus.cc:2454] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8985aa9192844a3686933c8fb7119e57 in term 0.
I20260806 18:32:14.631270 26271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d358055e3a54532b929d7365ee71cc9" candidate_uuid: "8985aa9192844a3686933c8fb7119e57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e17c9b996364dda9f11c000df34afca" is_pre_election: true
I20260806 18:32:14.631497 25991 leader_election.cc:302] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b57b4d2cc004cc89a05cef83df994e3, 8985aa9192844a3686933c8fb7119e57; no voters:
I20260806 18:32:14.631553 26271 raft_consensus.cc:2454] T 8d358055e3a54532b929d7365ee71cc9 P 3e17c9b996364dda9f11c000df34afca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8985aa9192844a3686933c8fb7119e57 in term 0.
I20260806 18:32:14.631762 26571 raft_consensus.cc:2781] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:32:14.631846 26571 raft_consensus.cc:482] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:32:14.631878 26571 raft_consensus.cc:3037] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:32:14.633446 26571 raft_consensus.cc:504] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8985aa9192844a3686933c8fb7119e57" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39757 } } peers { permanent_uuid: "0b57b4d2cc004cc89a05cef83df994e3" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35021 } } peers { permanent_uuid: "3e17c9b996364dda9f11c000df34afca" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45417 } }
I20260806 18:32:14.633798 26571 leader_election.cc:288] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [CANDIDATE]: Term 1 election: Requested vote from peers 0b57b4d2cc004cc89a05cef83df994e3 (127.7.67.195:35021), 3e17c9b996364dda9f11c000df34afca (127.7.67.194:45417)
I20260806 18:32:14.634238 26271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d358055e3a54532b929d7365ee71cc9" candidate_uuid: "8985aa9192844a3686933c8fb7119e57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e17c9b996364dda9f11c000df34afca"
I20260806 18:32:14.634373 26271 raft_consensus.cc:3037] T 8d358055e3a54532b929d7365ee71cc9 P 3e17c9b996364dda9f11c000df34afca [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:32:14.634454 26459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d358055e3a54532b929d7365ee71cc9" candidate_uuid: "8985aa9192844a3686933c8fb7119e57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b57b4d2cc004cc89a05cef83df994e3"
I20260806 18:32:14.634591 26459 raft_consensus.cc:3037] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:32:14.635666 26459 raft_consensus.cc:2454] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8985aa9192844a3686933c8fb7119e57 in term 1.
I20260806 18:32:14.635692 26271 raft_consensus.cc:2454] T 8d358055e3a54532b929d7365ee71cc9 P 3e17c9b996364dda9f11c000df34afca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8985aa9192844a3686933c8fb7119e57 in term 1.
I20260806 18:32:14.635960 25991 leader_election.cc:302] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e17c9b996364dda9f11c000df34afca, 8985aa9192844a3686933c8fb7119e57; no voters:
I20260806 18:32:14.636142 26571 raft_consensus.cc:2781] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:32:14.636413 26571 raft_consensus.cc:686] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [term 1 LEADER]: Becoming Leader. State: Replica: 8985aa9192844a3686933c8fb7119e57, State: Running, Role: LEADER
I20260806 18:32:14.636745 26571 consensus_queue.cc:237] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8985aa9192844a3686933c8fb7119e57" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39757 } } peers { permanent_uuid: "0b57b4d2cc004cc89a05cef83df994e3" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35021 } } peers { permanent_uuid: "3e17c9b996364dda9f11c000df34afca" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45417 } }
I20260806 18:32:14.639916 25865 catalog_manager.cc:5495] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8985aa9192844a3686933c8fb7119e57 (127.7.67.193). New cstate: current_term: 1 leader_uuid: "8985aa9192844a3686933c8fb7119e57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8985aa9192844a3686933c8fb7119e57" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b57b4d2cc004cc89a05cef83df994e3" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e17c9b996364dda9f11c000df34afca" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45417 } health_report { overall_health: UNKNOWN } } }
I20260806 18:32:14.727456 26556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:14.723339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51712 (local address 127.7.67.193:39757)
0806 18:32:14.723492 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:14.723498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:14.723516 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:32:14.723639 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:14.723643 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:14.723755 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:14.723883 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:32:14.724107 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.724896 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.725779 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:14.726108 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:14.726208 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:14.726467 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:32:14.727095 (+ 628us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:32:14.727213 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:14.727245 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 18:32:14.727307 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:32:14.734122 26271 raft_consensus.cc:1261] T 8d358055e3a54532b929d7365ee71cc9 P 3e17c9b996364dda9f11c000df34afca [term 1 FOLLOWER]: Refusing update from remote peer 8985aa9192844a3686933c8fb7119e57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:32:14.734120 26459 raft_consensus.cc:1261] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3 [term 1 FOLLOWER]: Refusing update from remote peer 8985aa9192844a3686933c8fb7119e57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:32:14.734947 26589 consensus_queue.cc:1035] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e17c9b996364dda9f11c000df34afca" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:32:14.735167 26571 consensus_queue.cc:1035] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b57b4d2cc004cc89a05cef83df994e3" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 35021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:32:14.743009 26593 mvcc.cc:205] Tried to move back new op lower bound from 7315624487862910976 to 7315624487474855936. Current Snapshot: MvccSnapshot[applied={T|T < 7315624487862910976}]
I20260806 18:32:14.744303 26599 mvcc.cc:205] Tried to move back new op lower bound from 7315624487862910976 to 7315624487474855936. Current Snapshot: MvccSnapshot[applied={T|T < 7315624487862910976}]
I20260806 18:32:14.747005 26594 mvcc.cc:205] Tried to move back new op lower bound from 7315624487862910976 to 7315624487474855936. Current Snapshot: MvccSnapshot[applied={T|T < 7315624487862910976}]
I20260806 18:32:14.754166 25864 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:32:14.754366 25864 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:32:14.757192 25864 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9410d1624c814f97b693e40b7b7b4547: Sending DeleteTablet for 3 replicas of tablet 8d358055e3a54532b929d7365ee71cc9
I20260806 18:32:14.758162 26036 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d358055e3a54532b929d7365ee71cc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:32:14 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:51696
I20260806 18:32:14.758172 26235 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d358055e3a54532b929d7365ee71cc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:32:14 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:47916
I20260806 18:32:14.758951 26431 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d358055e3a54532b929d7365ee71cc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:32:14 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:48272
I20260806 18:32:14.759163 26606 tablet_replica.cc:330] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57: stopping tablet replica
I20260806 18:32:14.759546 26606 raft_consensus.cc:2229] T 8d358055e3a54532b929d7365ee71cc9 P 8985aa9192844a3686933c8fb7119e57 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:32:14.760154 26609 tablet_replica.cc:330] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3: stopping tablet replica
I20260806 18:32:14.760257 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 25920
I20260806 18:32:14.760483 26609 raft_consensus.cc:2229] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:32:14.760845 26609 raft_consensus.cc:2258] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:32:14.765067 26609 ts_tablet_manager.cc:1905] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:32:14.768254 26609 ts_tablet_manager.cc:1918] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:32:14.768388 26609 log.cc:1196] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/8d358055e3a54532b929d7365ee71cc9
I20260806 18:32:14.768834 26609 ts_tablet_manager.cc:1939] T 8d358055e3a54532b929d7365ee71cc9 P 0b57b4d2cc004cc89a05cef83df994e3: Deleting consensus metadata
I20260806 18:32:14.770316 25845 catalog_manager.cc:4841] TS 0b57b4d2cc004cc89a05cef83df994e3 (127.7.67.195:35021): tablet 8d358055e3a54532b929d7365ee71cc9 (table test-table [id=6d654d22e2874492ae4dc6c56736244f]) successfully deleted
I20260806 18:32:14.772341 26608 tablet_replica.cc:330] T 8d358055e3a54532b929d7365ee71cc9 P 3e17c9b996364dda9f11c000df34afca: stopping tablet replica
W20260806 18:32:14.772312 25845 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.193:39757 (error 108)
I20260806 18:32:14.772689 26608 raft_consensus.cc:2229] T 8d358055e3a54532b929d7365ee71cc9 P 3e17c9b996364dda9f11c000df34afca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:32:14.773077 26608 raft_consensus.cc:2258] T 8d358055e3a54532b929d7365ee71cc9 P 3e17c9b996364dda9f11c000df34afca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:32:14.773487 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 26150
I20260806 18:32:14.773645 26558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:14.773306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:39757 (local address 127.0.0.1:51726)
0806 18:32:14.773486 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:14.773553 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:39757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260806 18:32:14.777859 25845 catalog_manager.cc:4584] TS 8985aa9192844a3686933c8fb7119e57 (127.7.67.193:39757): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d358055e3a54532b929d7365ee71cc9: Network error: Client connection negotiation failed: client connection to 127.7.67.193:39757: connect: Connection refused (error 111)
I20260806 18:32:14.784344 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 26337
I20260806 18:32:14.784786 26558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:14.784479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:45417 (local address 127.0.0.1:47926)
0806 18:32:14.784651 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:14.784700 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.194:45417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260806 18:32:14.806406 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 25818
I20260806 18:32:14.820508 7439 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 18:32:15.165308 7439 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 18:32:15.165439 7439 mini_ranger.cc:66] Stopping Ranger...
I20260806 18:32:15.510723 7439 mini_ranger.cc:68] Stopped Ranger
2026-08-06 18:32:15.510 UTC [24481] LOG: received smart shutdown request
2026-08-06 18:32:15.515 UTC [24481] LOG: background worker "logical replication launcher" (PID 24493) exited with exit code 1
2026-08-06 18:32:15.515 UTC [24488] LOG: shutting down
2026-08-06 18:32:15.664 UTC [24481] LOG: database system is shut down
2026-08-06T18:32:15Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49292 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:32:15 dist-test-slave-5bn2 krb5kdc[26622](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:32:15 dist-test-slave-5bn2 krb5kdc[26622](info): set up 2 sockets
Aug 06 18:32:15 dist-test-slave-5bn2 krb5kdc[26622](info): commencing operation
krb5kdc: starting...
W20260806 18:32:16.891875 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:32:16 dist-test-slave-5bn2 krb5kdc[26622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:32:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:32:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:32:17.015851 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34963
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:35743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:34963,127.7.67.253:34765,127.7.67.252:45359
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:32:17.130326 26644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:17.130621 26644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:17.130663 26644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:17.134680 26644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:32:17.134749 26644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:17.134768 26644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:17.134784 26644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:32:17.134797 26644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:32:17.140260 26644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:35743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:34963,127.7.67.253:34765,127.7.67.252:45359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:17.141381 26644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:17.142442 26644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:17.150769 26655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:17.150761 26656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:17.150766 26660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:17.151098 26644 server_base.cc:1034] running on GCE node
I20260806 18:32:17.151750 26644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:17.152849 26644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:17.154006 26644 hybrid_clock.cc:648] HybridClock initialized: now 1786041137153980 us; error 35 us; skew 500 ppm
Aug 06 18:32:17 dist-test-slave-5bn2 krb5kdc[26622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:17.156823 26644 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:32:17.157706 26644 webserver.cc:460] Webserver started at http://127.7.67.254:44243/ using document root <none> and password file <none>
I20260806 18:32:17.158257 26644 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:17.158324 26644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:17.158516 26644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:32:17.160576 26644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "7c6a4c718a3b402d8010a861ed1ae42d"
format_stamp: "Formatted at 2026-08-06 18:32:17 on dist-test-slave-5bn2"
server_key: "345fa73708be20bb1ddce0039c417d28"
server_key_iv: "b84f383b9f085c56c4b215e2588f5620"
server_key_version: "encryptionkey@0"
I20260806 18:32:17.161131 26644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "7c6a4c718a3b402d8010a861ed1ae42d"
format_stamp: "Formatted at 2026-08-06 18:32:17 on dist-test-slave-5bn2"
server_key: "345fa73708be20bb1ddce0039c417d28"
server_key_iv: "b84f383b9f085c56c4b215e2588f5620"
server_key_version: "encryptionkey@0"
I20260806 18:32:17.164798 26644 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 18:32:17.167119 26668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:17.168288 26644 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:32:17.168404 26644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "7c6a4c718a3b402d8010a861ed1ae42d"
format_stamp: "Formatted at 2026-08-06 18:32:17 on dist-test-slave-5bn2"
server_key: "345fa73708be20bb1ddce0039c417d28"
server_key_iv: "b84f383b9f085c56c4b215e2588f5620"
server_key_version: "encryptionkey@0"
I20260806 18:32:17.168504 26644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:17.185138 26644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:17.188153 26644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:17.188336 26644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:17.195218 26644 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:34963
I20260806 18:32:17.195261 26746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:34963 every 8 connection(s)
I20260806 18:32:17.196158 26644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:32:17.198894 26747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:32:17.202786 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 26644
I20260806 18:32:17.202930 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:32:17.202659 26747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:32:17.203200 7439 external_mini_cluster.cc:1428] Setting key 1e758d1d22940a9137f6ca29b66b5702
WARNING: no policy specified for kudu/127.7.67.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.253@KRBTEST.COM" created.
Aug 06 18:32:17 dist-test-slave-5bn2 krb5kdc[26622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:17.221125 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.205054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34963 (local address 127.0.0.1:44386)
0806 18:32:17.205598 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.205657 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:17.206650 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:17.207329 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:17.207335 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:17.207758 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:17.208146 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:17.208166 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.209255 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.209258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:17.210420 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:17.210432 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.211181 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.211186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:17.211367 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.211960 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:17.211981 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:17.215901 (+ 3920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:17.218099 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:17.218107 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:17.218115 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:17.218349 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:17.218704 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:17.218706 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:17.218708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:17.218812 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:17.219591 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:17.219597 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:17.219731 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 18:32:17.219883 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 18:32:17.220311 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":127,"thread_start_us":128,"threads_started":1}
I20260806 18:32:17.221133 26752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.205823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44386 (local address 127.7.67.254:34963)
0806 18:32:17.206067 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:17.206076 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:17.206110 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:32:17.206951 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:17.206955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:17.207131 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:17.207282 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:17.208289 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.209127 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.210561 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.211012 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.211154 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.216048 (+ 4894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:17.216054 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:17.216065 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:17.216086 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:17.217857 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.218463 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.218469 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.218470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.218616 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.218927 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.218930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.218931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.219450 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:17.219583 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:17.220006 (+ 423us) server_negotiation.cc:299] Negotiation successful
0806 18:32:17.220251 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":157,"thread_start_us":70,"threads_started":1}
I20260806 18:32:17.223857 26747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:32:17.224628 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.224354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:34765 (local address 127.0.0.1:60540)
0806 18:32:17.224494 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.224567 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.253:34765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 18:32:17.224807 26673 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.67.253:34765: connect: Connection refused (error 111)
I20260806 18:32:17.225652 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.225443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:34765 (local address 127.0.0.1:60548)
0806 18:32:17.225552 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.225598 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.253:34765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 18:32:17.225781 26747 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.253:34765: Network error: Client connection negotiation failed: client connection to 127.7.67.253:34765: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.253.keytab.
Entry for principal kudu/127.7.67.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.253.keytab.
I20260806 18:32:17.232463 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.253:34765
--webserver_interface=127.7.67.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.253.keytab
--principal=kudu/127.7.67.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:35743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:34963,127.7.67.253:34765,127.7.67.252:45359
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:32:17.274987 26747 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } attempt: 1
I20260806 18:32:17.275867 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.275545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:34765 (local address 127.0.0.1:60558)
0806 18:32:17.275703 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.275767 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.253:34765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 18:32:17.276571 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.276355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:34765 (local address 127.0.0.1:60560)
0806 18:32:17.276457 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.276503 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.253:34765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 18:32:17.276685 26747 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.253:34765: Network error: Client connection negotiation failed: client connection to 127.7.67.253:34765: connect: Connection refused (error 111)
I20260806 18:32:17.344867 26747 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } attempt: 2
I20260806 18:32:17.345788 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.345448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:34765 (local address 127.0.0.1:60564)
0806 18:32:17.345632 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.345703 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.253:34765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 18:32:17.346542 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.346308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:34765 (local address 127.0.0.1:60574)
0806 18:32:17.346428 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.346479 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.253:34765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 18:32:17.346439 26757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:17.346668 26747 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.253:34765: Network error: Client connection negotiation failed: client connection to 127.7.67.253:34765: connect: Connection refused (error 111)
W20260806 18:32:17.346756 26757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:17.346827 26757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:17.350833 26757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:32:17.350919 26757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:17.350955 26757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:17.350986 26757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:32:17.351015 26757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:32:17.356220 26757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:35743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:34963,127.7.67.253:34765,127.7.67.252:45359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.253.keytab
--principal=kudu/127.7.67.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.253:34765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.67.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:17.357272 26757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:17.358454 26757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:17.366793 26764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:17.366839 26765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:17.366793 26767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:17.367132 26757 server_base.cc:1034] running on GCE node
I20260806 18:32:17.367738 26757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:17.368831 26757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:17.370028 26757 hybrid_clock.cc:648] HybridClock initialized: now 1786041137370006 us; error 30 us; skew 500 ppm
Aug 06 18:32:17 dist-test-slave-5bn2 krb5kdc[26622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:17.372848 26757 init.cc:377] Logged in from keytab as kudu/127.7.67.253@KRBTEST.COM (short username kudu)
I20260806 18:32:17.373771 26757 webserver.cc:460] Webserver started at http://127.7.67.253:36443/ using document root <none> and password file <none>
I20260806 18:32:17.374313 26757 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:17.374374 26757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:17.374593 26757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:32:17.376703 26757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-1/data/instance:
uuid: "4f4821bd9f444fd799474b583f30c95a"
format_stamp: "Formatted at 2026-08-06 18:32:17 on dist-test-slave-5bn2"
server_key: "12bc87b1c64ee47d088cf06aa5b15987"
server_key_iv: "32224fa6bcc76debb2928aa8396fb973"
server_key_version: "encryptionkey@0"
I20260806 18:32:17.377242 26757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-1/wal/instance:
uuid: "4f4821bd9f444fd799474b583f30c95a"
format_stamp: "Formatted at 2026-08-06 18:32:17 on dist-test-slave-5bn2"
server_key: "12bc87b1c64ee47d088cf06aa5b15987"
server_key_iv: "32224fa6bcc76debb2928aa8396fb973"
server_key_version: "encryptionkey@0"
I20260806 18:32:17.381116 26757 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260806 18:32:17.383522 26780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:17.384706 26757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:32:17.384831 26757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-1/wal
uuid: "4f4821bd9f444fd799474b583f30c95a"
format_stamp: "Formatted at 2026-08-06 18:32:17 on dist-test-slave-5bn2"
server_key: "12bc87b1c64ee47d088cf06aa5b15987"
server_key_iv: "32224fa6bcc76debb2928aa8396fb973"
server_key_version: "encryptionkey@0"
I20260806 18:32:17.384936 26757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:17.398874 26757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:17.401973 26757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:17.402151 26757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:17.408980 26757 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.253:34765
I20260806 18:32:17.409004 26854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.253:34765 every 8 connection(s)
I20260806 18:32:17.409937 26757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-1/data/info.pb
I20260806 18:32:17.412719 26855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:32:17.416566 26855 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:32:17.419898 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 26757
I20260806 18:32:17.420042 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-1/wal/instance
I20260806 18:32:17.420336 7439 external_mini_cluster.cc:1428] Setting key 3896ad9bec64ce5722a6da408f9b73ad
Aug 06 18:32:17 dist-test-slave-5bn2 krb5kdc[26622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.253@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.252@KRBTEST.COM; defaulting to no policy
I20260806 18:32:17.430287 26752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.418403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44398 (local address 127.7.67.254:34963)
0806 18:32:17.418582 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:17.418587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:17.419456 (+ 869us) server_negotiation.cc:407] Connection header received
0806 18:32:17.420541 (+ 1085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:17.420544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:17.420597 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:17.420668 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:17.422130 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.422660 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.423839 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.424095 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.424188 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.426749 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:17.426752 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:17.426754 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:17.426780 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:17.428148 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.428757 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.428760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.428762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.428838 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.429223 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.429225 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.429226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.429416 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:17.429491 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:17.430026 (+ 535us) server_negotiation.cc:299] Negotiation successful
0806 18:32:17.430158 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 18:32:17.431130 26857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.418792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34963 (local address 127.0.0.1:44398)
0806 18:32:17.419302 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.419345 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:17.420314 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:17.420870 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:17.420879 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:17.421323 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:17.421988 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:17.422007 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.422793 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.422796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:17.423708 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:17.423719 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.424246 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.424250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:17.424411 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.424971 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:17.424989 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:17.426560 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:17.428276 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:17.428282 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:17.428298 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:17.428660 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:17.428919 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:17.428922 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:17.428924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:17.429127 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:17.429504 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:17.429510 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:17.429789 (+ 279us) client_negotiation.cc:769] Sending connection context
0806 18:32:17.429963 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 18:32:17.430259 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":127,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.252@KRBTEST.COM" created.
I20260806 18:32:17.432633 26855 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:32:17 dist-test-slave-5bn2 krb5kdc[26622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.253@KRBTEST.COM for kudu/127.7.67.253@KRBTEST.COM
I20260806 18:32:17.437808 26747 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } attempt: 3
I20260806 18:32:17.441864 26857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.433156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:34765 (local address 127.0.0.1:60584)
0806 18:32:17.433263 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.433278 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:17.433359 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:17.433940 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:17.433943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:17.434105 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:17.434181 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:17.434186 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.435168 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.435172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:17.435692 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:17.435700 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.436256 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.436259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:17.436343 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.436823 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:17.436833 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:17.437997 (+ 1164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:17.440300 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:17.440303 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:17.440305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:17.440547 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:17.440874 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:17.440876 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:17.440877 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:17.440952 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:17.441389 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:17.441393 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:17.441505 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 18:32:17.441592 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 18:32:17.441729 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 18:32:17.442363 26862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.433243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60584 (local address 127.7.67.253:34765)
0806 18:32:17.433619 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:17.433628 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:17.433646 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:32:17.433677 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:17.433682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:17.433827 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:17.433944 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:17.434280 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.435059 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.435791 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.436152 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.436241 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.438141 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:17.438152 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:17.438167 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:17.438194 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:17.440189 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.440672 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.440679 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.440681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.440783 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.441055 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.441057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.441058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.441291 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:17.441395 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:17.441696 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 18:32:17.441852 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":118,"threads_started":1}
I20260806 18:32:17.444332 26855 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:32:17 dist-test-slave-5bn2 krb5kdc[26622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for kudu/127.7.67.253@KRBTEST.COM
I20260806 18:32:17.445086 26857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.444802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:45359 (local address 127.0.0.1:37608)
0806 18:32:17.444928 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.445013 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:45359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 18:32:17.445314 26781 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.67.252:45359: connect: Connection refused (error 111)
I20260806 18:32:17.446555 26857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.446304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:45359 (local address 127.0.0.1:37622)
0806 18:32:17.446423 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.446476 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:45359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 18:32:17.446712 26855 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.252:45359: Network error: Client connection negotiation failed: client connection to 127.7.67.252:45359: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.252.keytab.
Entry for principal kudu/127.7.67.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.252.keytab.
I20260806 18:32:17.448200 26863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.438465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60588 (local address 127.7.67.253:34765)
0806 18:32:17.438782 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:17.438788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:17.438799 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:32:17.438826 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:17.438827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:17.438890 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:17.438972 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:17.439843 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.440380 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.441445 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.441773 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.441867 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.444949 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:17.444952 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:17.444955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:17.444978 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:17.446470 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.446991 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.446994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.447002 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.447077 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.447375 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.447378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.447380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.447600 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:17.447698 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:17.447990 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 18:32:17.448086 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":122,"threads_started":1}
I20260806 18:32:17.448251 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.438265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:34765 (local address 127.0.0.1:60588)
0806 18:32:17.438389 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.438404 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:17.438495 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:17.439022 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:17.439025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:17.439208 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:17.439731 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:17.439741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.440575 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.440579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:17.441320 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:17.441332 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.441910 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.441914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:17.442046 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.443282 (+ 1236us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:17.443296 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:17.444777 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:17.446576 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:17.446579 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:17.446580 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:17.446913 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:17.447149 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:17.447152 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:17.447154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:17.447252 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:17.447707 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:17.447711 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:17.447825 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 18:32:17.447937 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 18:32:17.448094 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 18:32:17.448359 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.252:45359
--webserver_interface=127.7.67.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.252.keytab
--principal=kudu/127.7.67.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:35743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:34963,127.7.67.253:34765,127.7.67.252:45359
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:32:17.448902 26747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:32:17.449558 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.449325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:45359 (local address 127.0.0.1:37628)
0806 18:32:17.449451 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.449499 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:45359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 18:32:17.450302 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.449968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:45359 (local address 127.0.0.1:37632)
0806 18:32:17.450184 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.450237 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:45359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
W20260806 18:32:17.450419 26747 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.252:45359: Network error: Client connection negotiation failed: client connection to 127.7.67.252:45359: connect: Connection refused (error 111)
I20260806 18:32:17.481566 26747 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } attempt: 1
I20260806 18:32:17.482409 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.482098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:45359 (local address 127.0.0.1:37638)
0806 18:32:17.482260 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.482325 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:45359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 18:32:17.483278 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.482995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:45359 (local address 127.0.0.1:37650)
0806 18:32:17.483145 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.483208 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:45359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 18:32:17.483418 26747 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.252:45359: Network error: Client connection negotiation failed: client connection to 127.7.67.252:45359: connect: Connection refused (error 111)
I20260806 18:32:17.495952 26855 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } attempt: 1
I20260806 18:32:17.496830 26857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.496470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:45359 (local address 127.0.0.1:37656)
0806 18:32:17.496648 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.496730 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:45359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 18:32:17.497581 26857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.497346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:45359 (local address 127.0.0.1:37658)
0806 18:32:17.497457 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.497513 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:45359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 18:32:17.497731 26855 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.252:45359: Network error: Client connection negotiation failed: client connection to 127.7.67.252:45359: connect: Connection refused (error 111)
I20260806 18:32:17.558604 26747 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } attempt: 2
I20260806 18:32:17.559572 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.559249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:45359 (local address 127.0.0.1:37668)
0806 18:32:17.559421 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.559489 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:45359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 18:32:17.560271 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.560050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:45359 (local address 127.0.0.1:37678)
0806 18:32:17.560155 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.560211 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:45359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 18:32:17.560387 26747 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.252:45359: Network error: Client connection negotiation failed: client connection to 127.7.67.252:45359: connect: Connection refused (error 111)
W20260806 18:32:17.563697 26866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:17.563958 26866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:17.564013 26866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 18:32:17.565891 26855 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } attempt: 2
I20260806 18:32:17.566665 26857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.566349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:45359 (local address 127.0.0.1:37684)
0806 18:32:17.566520 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.566585 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:45359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 18:32:17.567480 26857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.567248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:45359 (local address 127.0.0.1:37698)
0806 18:32:17.567363 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.567418 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:45359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 18:32:17.567603 26855 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.252:45359: Network error: Client connection negotiation failed: client connection to 127.7.67.252:45359: connect: Connection refused (error 111)
W20260806 18:32:17.568310 26866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:32:17.568362 26866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:17.568389 26866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:17.568420 26866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:32:17.568449 26866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:32:17.573892 26866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:35743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:34963,127.7.67.253:34765,127.7.67.252:45359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.252.keytab
--principal=kudu/127.7.67.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.252:45359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.67.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:17.575018 26866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:17.576035 26866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:17.584358 26875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:17.584360 26871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:17.584360 26872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:17.584559 26866 server_base.cc:1034] running on GCE node
I20260806 18:32:17.585310 26866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:17.586392 26866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:17.587546 26866 hybrid_clock.cc:648] HybridClock initialized: now 1786041137587525 us; error 33 us; skew 500 ppm
Aug 06 18:32:17 dist-test-slave-5bn2 krb5kdc[26622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:17.590297 26866 init.cc:377] Logged in from keytab as kudu/127.7.67.252@KRBTEST.COM (short username kudu)
I20260806 18:32:17.591255 26866 webserver.cc:460] Webserver started at http://127.7.67.252:36701/ using document root <none> and password file <none>
I20260806 18:32:17.591804 26866 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:17.591871 26866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:17.592064 26866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:32:17.594130 26866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-2/data/instance:
uuid: "e075aa3262494c72ae0fcd1d05734eb0"
format_stamp: "Formatted at 2026-08-06 18:32:17 on dist-test-slave-5bn2"
server_key: "7a3350934f4321d2033c51e0fbfa0bfd"
server_key_iv: "36dbcada08ac4d263fc7fa0aabcbf308"
server_key_version: "encryptionkey@0"
I20260806 18:32:17.594684 26866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-2/wal/instance:
uuid: "e075aa3262494c72ae0fcd1d05734eb0"
format_stamp: "Formatted at 2026-08-06 18:32:17 on dist-test-slave-5bn2"
server_key: "7a3350934f4321d2033c51e0fbfa0bfd"
server_key_iv: "36dbcada08ac4d263fc7fa0aabcbf308"
server_key_version: "encryptionkey@0"
I20260806 18:32:17.598253 26866 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:32:17.600633 26888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:17.601770 26866 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:32:17.601884 26866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-2/wal
uuid: "e075aa3262494c72ae0fcd1d05734eb0"
format_stamp: "Formatted at 2026-08-06 18:32:17 on dist-test-slave-5bn2"
server_key: "7a3350934f4321d2033c51e0fbfa0bfd"
server_key_iv: "36dbcada08ac4d263fc7fa0aabcbf308"
server_key_version: "encryptionkey@0"
I20260806 18:32:17.601979 26866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:17.615799 26866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:17.618952 26866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:17.619143 26866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:17.626081 26866 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.252:45359
I20260806 18:32:17.626096 26963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.252:45359 every 8 connection(s)
I20260806 18:32:17.627084 26866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-2/data/info.pb
I20260806 18:32:17.630084 26965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:32:17.634231 26965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:32:17.635650 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 26866
I20260806 18:32:17.635813 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786040973123132-7439-0/minicluster-data/master-2/wal/instance
I20260806 18:32:17.636107 7439 external_mini_cluster.cc:1428] Setting key 50197ab965690bf829167bcad1d021d7
Aug 06 18:32:17 dist-test-slave-5bn2 krb5kdc[26622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.252@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
Aug 06 18:32:17 dist-test-slave-5bn2 krb5kdc[26622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.252@KRBTEST.COM
I20260806 18:32:17.648020 26752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.636398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44410 (local address 127.7.67.254:34963)
0806 18:32:17.636565 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:17.636570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:17.637488 (+ 918us) server_negotiation.cc:407] Connection header received
0806 18:32:17.638546 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:17.638550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:17.638608 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:17.638677 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:17.639612 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.640118 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.641113 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.641327 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.641416 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.644179 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:17.644183 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:17.644184 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:17.644213 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:17.645739 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.646321 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.646325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.646327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.646417 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.646814 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.646817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.646819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.647076 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:17.647172 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:17.647718 (+ 546us) server_negotiation.cc:299] Negotiation successful
0806 18:32:17.647845 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:32:17.648983 26968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.636810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34963 (local address 127.0.0.1:44410)
0806 18:32:17.637313 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.637367 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:17.638326 (+ 959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:17.638742 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:17.638748 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:17.639152 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:17.639462 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:17.639477 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.640223 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.640227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:17.640987 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:17.640997 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.641476 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.641480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:17.641646 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.642237 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:17.642259 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:17.643987 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:17.645862 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:17.645872 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:17.645881 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:17.646218 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:17.646521 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:17.646525 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:17.646527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:17.646720 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:17.647185 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:17.647190 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:17.647472 (+ 282us) client_negotiation.cc:769] Sending connection context
0806 18:32:17.647664 (+ 192us) client_negotiation.cc:240] Negotiation successful
0806 18:32:17.648026 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":26,"thread_start_us":126,"threads_started":1}
I20260806 18:32:17.648991 26970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.637706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37710 (local address 127.7.67.252:45359)
0806 18:32:17.637988 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:17.637999 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:17.638037 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:32:17.638346 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:17.638363 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:17.638559 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:17.638748 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:17.639717 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.640579 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.641429 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.641797 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.641905 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.644580 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:17.644586 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:17.644602 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:17.644628 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:17.646252 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.646734 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.646738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.646740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.646825 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.647115 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.647118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.647120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.647463 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:17.647613 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:17.647895 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 18:32:17.648086 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":188,"thread_start_us":92,"threads_started":1}
I20260806 18:32:17.650941 26965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:32:17 dist-test-slave-5bn2 krb5kdc[26622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.252@KRBTEST.COM for kudu/127.7.67.253@KRBTEST.COM
Aug 06 18:32:17 dist-test-slave-5bn2 krb5kdc[26622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:17.658768 26855 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } attempt: 3
I20260806 18:32:17.659528 26747 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } attempt: 3
I20260806 18:32:17.660210 26863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.651627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60600 (local address 127.7.67.253:34765)
0806 18:32:17.651762 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:17.651766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:17.651782 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:32:17.651833 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:17.651835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:17.651882 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:17.651958 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:17.652528 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.653360 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.654136 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.654400 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.654496 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.656677 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:17.656681 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:17.656684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:17.656718 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:17.658316 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.658772 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.658776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.658778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.658935 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.659268 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.659271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.659273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.659498 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:17.659567 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:17.659928 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 18:32:17.660049 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:32:17.660691 26968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.651512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:34765 (local address 127.0.0.1:60600)
0806 18:32:17.651675 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.651689 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:17.651766 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:17.651982 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:17.651985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:17.652170 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:17.652309 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:17.652374 (+ 65us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.653492 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.653496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:17.654015 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:17.654024 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.654519 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.654523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:17.654632 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.655280 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:17.655297 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:17.656521 (+ 1224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:17.658435 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:17.658438 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:17.658441 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:17.658671 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:17.659037 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:17.659040 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:17.659042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:17.659142 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:17.659642 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:17.659644 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:17.659746 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 18:32:17.660403 (+ 657us) client_negotiation.cc:240] Negotiation successful
0806 18:32:17.660537 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260806 18:32:17.661547 26965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:32:17.662309 26752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.651522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44414 (local address 127.7.67.254:34963)
0806 18:32:17.651627 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:17.651631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:17.651808 (+ 177us) server_negotiation.cc:407] Connection header received
0806 18:32:17.651843 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:17.651845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:17.651882 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:17.651964 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:17.652719 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.653435 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.654176 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.654397 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.654494 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.657189 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:17.657192 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:17.657194 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:17.657218 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:17.658617 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.659113 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.659117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.659119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.659221 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.660385 (+ 1164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.660389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.660391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.660634 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:17.660877 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:17.661651 (+ 774us) server_negotiation.cc:299] Negotiation successful
0806 18:32:17.661781 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 06 18:32:17 dist-test-slave-5bn2 krb5kdc[26622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.253@KRBTEST.COM for kudu/127.7.67.252@KRBTEST.COM
Aug 06 18:32:17 dist-test-slave-5bn2 krb5kdc[26622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for kudu/127.7.67.252@KRBTEST.COM
I20260806 18:32:17.668411 26970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.659342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37712 (local address 127.7.67.252:45359)
0806 18:32:17.659478 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:17.659483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:17.659499 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:32:17.659617 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:17.659621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:17.659669 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:17.659746 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:17.660149 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.661107 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.662064 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.662453 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.662563 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.665145 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:17.665149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:17.665152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:17.665174 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:17.666591 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.667074 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.667077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.667079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.667161 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.667493 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.667496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.667498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.667687 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:17.667784 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:17.668137 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 18:32:17.668249 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":59}
I20260806 18:32:17.668421 26857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.659228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:45359 (local address 127.0.0.1:37712)
0806 18:32:17.659433 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.659448 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:17.659528 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:17.659762 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:17.659765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:17.659933 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:17.660025 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:17.660032 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.661215 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.661219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:17.661927 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:17.661938 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.662699 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.662703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:17.662782 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.663403 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:17.663418 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:17.665001 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:17.666696 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:17.666699 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:17.666701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:17.666964 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:17.667255 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:17.667258 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:17.667260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:17.667361 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:17.667808 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:17.667812 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:17.667938 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 18:32:17.668072 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 18:32:17.668224 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
Aug 06 18:32:17 dist-test-slave-5bn2 krb5kdc[26622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.252@KRBTEST.COM for kudu/127.7.67.252@KRBTEST.COM
Aug 06 18:32:17 dist-test-slave-5bn2 krb5kdc[26622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.253@KRBTEST.COM
I20260806 18:32:17.672435 26855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a: Bootstrap starting.
I20260806 18:32:17.672935 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.659934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:45359 (local address 127.0.0.1:37724)
0806 18:32:17.660120 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.660134 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:17.660217 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:17.660623 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:17.660626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:17.660798 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:17.660908 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:17.660917 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.662796 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.662800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:17.663540 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:17.663551 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.664131 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.664135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:17.664220 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.664735 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:17.664752 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:17.666831 (+ 2079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:17.670627 (+ 3796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:17.670631 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:17.670634 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:17.670955 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:17.671315 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:17.671331 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:17.671333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:17.671687 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:17.672431 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:17.672434 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:17.672546 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 18:32:17.672644 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 18:32:17.672780 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260806 18:32:17.673235 26971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.659983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37724 (local address 127.7.67.252:45359)
0806 18:32:17.660362 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:17.660381 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:17.660394 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:32:17.660427 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:17.660430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:17.660471 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:17.661040 (+ 569us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:17.661142 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.662665 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.663670 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.663988 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.667164 (+ 3176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.667287 (+ 123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:17.667291 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:17.667303 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:17.667334 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:17.670458 (+ 3124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.671086 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.671089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.671091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.671189 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.671829 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.671833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.671835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.672283 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:17.672411 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:17.672766 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 18:32:17.672884 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":300,"thread_start_us":113,"threads_started":1}
I20260806 18:32:17.673421 26968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.662081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:45359 (local address 127.0.0.1:37726)
0806 18:32:17.662587 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:17.662601 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:17.662665 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:17.664186 (+ 1521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:17.664189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:17.664342 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:17.664419 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:17.664425 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.665350 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.665353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:17.666062 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:17.666072 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.667246 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.667249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:17.667320 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.667725 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:17.667739 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:17.669058 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:17.671066 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:17.671069 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:17.671071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:17.671288 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:17.671537 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:17.671540 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:17.671541 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:17.671638 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:17.672026 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:17.672030 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:17.673027 (+ 997us) client_negotiation.cc:769] Sending connection context
0806 18:32:17.673139 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 18:32:17.673267 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440}
I20260806 18:32:17.673758 26972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.662129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37726 (local address 127.7.67.252:45359)
0806 18:32:17.662387 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:17.662985 (+ 598us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:17.663006 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:32:17.663039 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:17.663042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:17.663083 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:17.664070 (+ 987us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:17.664539 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.665250 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.666693 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.667063 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.667148 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.669179 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:17.669182 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:17.669185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:17.669206 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:17.670953 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.671383 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.671385 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.671387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.671458 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.671720 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.671723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.671724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.671901 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:17.671992 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:17.673493 (+ 1501us) server_negotiation.cc:299] Negotiation successful
0806 18:32:17.673602 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":109,"threads_started":1}
I20260806 18:32:17.675390 26863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.663776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60616 (local address 127.7.67.253:34765)
0806 18:32:17.663904 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:17.663909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:17.663941 (+ 32us) server_negotiation.cc:407] Connection header received
0806 18:32:17.663980 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:17.663983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:17.664026 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:17.664100 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:17.665016 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.665694 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.666804 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.667133 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.667202 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.671116 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:17.671120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:17.671123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:17.671150 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:17.672873 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.674023 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.674027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.674029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.674126 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.674437 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.674441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.674442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.674639 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:17.674831 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:17.675080 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 18:32:17.675205 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 18:32:17.675508 26855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a: Neither blocks nor log segments found. Creating new log.
I20260806 18:32:17.676483 26855 log.cc:824] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:17.676869 26747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d: Bootstrap starting.
I20260806 18:32:17.678908 26855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a: No bootstrap required, opened a new log
I20260806 18:32:17.679769 26747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d: Neither blocks nor log segments found. Creating new log.
I20260806 18:32:17.680639 26747 log.cc:824] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:17.682289 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:32:17.683051 26855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } } peers { permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } } peers { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } }
I20260806 18:32:17.683292 26855 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:32:17.683377 26855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f4821bd9f444fd799474b583f30c95a, State: Initialized, Role: FOLLOWER
I20260806 18:32:17.683445 26965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0: Bootstrap starting.
I20260806 18:32:17.683907 26855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } } peers { permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } } peers { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } }
I20260806 18:32:17.684448 26983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } } peers { permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } } peers { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } } }
I20260806 18:32:17.684593 26983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:32:17.685329 26855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:32:17.686183 26965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0: Neither blocks nor log segments found. Creating new log.
I20260806 18:32:17.687132 26965 log.cc:824] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:17.689742 26747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d: No bootstrap required, opened a new log
I20260806 18:32:17.690282 26965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0: No bootstrap required, opened a new log
I20260806 18:32:17.689502 26757 master_runner.cc:186] Error getting master registration for 127.7.67.254:34963: OK, instance_id { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" instance_seqno: 1786041137193381 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:32:17.692795 26757 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:32:17.693742 26747 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } } peers { permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } } peers { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } }
I20260806 18:32:17.694517 26747 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:32:17.694591 26747 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c6a4c718a3b402d8010a861ed1ae42d, State: Initialized, Role: FOLLOWER
I20260806 18:32:17.695098 26747 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } } peers { permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } } peers { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } }
I20260806 18:32:17.695596 27006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } } peers { permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } } peers { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } } }
I20260806 18:32:17.695796 27006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:32:17.696722 26747 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d [sys.catalog]: configured and running, proceeding with master startup.
W20260806 18:32:17.697360 26999 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 18:32:17.698681 26965 raft_consensus.cc:348] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } } peers { permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } } peers { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } }
I20260806 18:32:17.701692 26965 raft_consensus.cc:374] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:32:17.701826 26965 raft_consensus.cc:729] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e075aa3262494c72ae0fcd1d05734eb0, State: Initialized, Role: FOLLOWER
W20260806 18:32:17.701812 27020 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:32:17.701896 27020 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:32:17.702415 26965 consensus_queue.cc:260] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } } peers { permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } } peers { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } }
I20260806 18:32:17.704814 26965 sys_catalog.cc:564] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:32:17.699604 26972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.687242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37728 (local address 127.7.67.252:45359)
0806 18:32:17.687380 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:17.687385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:17.689933 (+ 2548us) server_negotiation.cc:407] Connection header received
0806 18:32:17.689980 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:17.689983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:17.690032 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:17.690123 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:17.691441 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.692221 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.693395 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.693785 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.693920 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.695592 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:17.695596 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:17.695598 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:17.695625 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:17.697211 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.697715 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.697719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.697720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.697816 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.698103 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.698107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.698108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.698327 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:17.698479 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:17.699302 (+ 823us) server_negotiation.cc:299] Negotiation successful
0806 18:32:17.699428 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260806 18:32:17.707688 26999 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:32:17.708146 26863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.689134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60618 (local address 127.7.67.253:34765)
0806 18:32:17.689234 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:17.689238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:17.699191 (+ 9953us) server_negotiation.cc:407] Connection header received
0806 18:32:17.699816 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:17.699820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:17.699875 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:17.699924 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:17.700727 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.701464 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.702445 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.702753 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.702822 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.704463 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:17.704467 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:17.704482 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:17.704510 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:17.706215 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.706714 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.706718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.706719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.706808 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.707150 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.707154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.707156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.707389 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:17.707498 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:17.707866 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 18:32:17.707984 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:32:17.704144 26644 master_runner.cc:186] Error getting master registration for 127.7.67.252:45359: OK, instance_id { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" instance_seqno: 1786041137624148 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:32:17.708508 26644 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:32:17.710832 26752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:17.688846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44426 (local address 127.7.67.254:34963)
0806 18:32:17.688956 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:17.688961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:17.691290 (+ 2329us) server_negotiation.cc:407] Connection header received
0806 18:32:17.691400 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:17.691403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:17.691449 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:17.691518 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:17.699841 (+ 8323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.700587 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.702705 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:17.703030 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:17.703097 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:17.707381 (+ 4284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:17.707385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:17.707387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:17.707414 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:17.708957 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.709416 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.709420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.709421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.709521 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:17.709810 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:17.709813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:17.709815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:17.710022 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:17.710154 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:17.710535 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 18:32:17.710671 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 18:32:17.711117 27022 sys_catalog.cc:455] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } } peers { permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } } peers { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } } }
I20260806 18:32:17.712100 27022 sys_catalog.cc:458] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 18:32:17.718338 27039 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:32:17.718511 27039 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:32:17.720892 26866 master_runner.cc:263] No leader master found from master e075aa3262494c72ae0fcd1d05734eb0
I20260806 18:32:17.720970 26866 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:32:17.732180 27022 raft_consensus.cc:482] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:32:17.732366 27022 raft_consensus.cc:504] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } } peers { permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } } peers { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } }
I20260806 18:32:17.733330 27022 leader_election.cc:288] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c6a4c718a3b402d8010a861ed1ae42d (127.7.67.254:34963), 4f4821bd9f444fd799474b583f30c95a (127.7.67.253:34765)
I20260806 18:32:17.734158 26817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e075aa3262494c72ae0fcd1d05734eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f4821bd9f444fd799474b583f30c95a" is_pre_election: true
I20260806 18:32:17.734484 26817 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e075aa3262494c72ae0fcd1d05734eb0 in term 0.
I20260806 18:32:17.735255 26704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e075aa3262494c72ae0fcd1d05734eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" is_pre_election: true
I20260806 18:32:17.735564 26704 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e075aa3262494c72ae0fcd1d05734eb0 in term 0.
I20260806 18:32:17.736052 26892 leader_election.cc:302] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c6a4c718a3b402d8010a861ed1ae42d, e075aa3262494c72ae0fcd1d05734eb0; no voters:
I20260806 18:32:17.736616 27022 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:32:17.736760 27022 raft_consensus.cc:482] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:32:17.736855 27022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:32:17.738279 27022 raft_consensus.cc:504] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } } peers { permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } } peers { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } }
I20260806 18:32:17.738720 27022 leader_election.cc:288] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [CANDIDATE]: Term 1 election: Requested vote from peers 7c6a4c718a3b402d8010a861ed1ae42d (127.7.67.254:34963), 4f4821bd9f444fd799474b583f30c95a (127.7.67.253:34765)
I20260806 18:32:17.739400 26817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e075aa3262494c72ae0fcd1d05734eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f4821bd9f444fd799474b583f30c95a"
I20260806 18:32:17.739527 26817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:32:17.741446 26817 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e075aa3262494c72ae0fcd1d05734eb0 in term 1.
I20260806 18:32:17.741845 26704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e075aa3262494c72ae0fcd1d05734eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c6a4c718a3b402d8010a861ed1ae42d"
I20260806 18:32:17.741977 26704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:32:17.743306 26704 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e075aa3262494c72ae0fcd1d05734eb0 in term 1.
I20260806 18:32:17.746202 26892 leader_election.cc:302] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c6a4c718a3b402d8010a861ed1ae42d, e075aa3262494c72ae0fcd1d05734eb0; no voters:
I20260806 18:32:17.746621 27022 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:32:17.746915 27022 raft_consensus.cc:686] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [term 1 LEADER]: Becoming Leader. State: Replica: e075aa3262494c72ae0fcd1d05734eb0, State: Running, Role: LEADER
I20260806 18:32:17.747332 27022 consensus_queue.cc:237] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } } peers { permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } } peers { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } }
I20260806 18:32:17.749518 27022 sys_catalog.cc:455] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e075aa3262494c72ae0fcd1d05734eb0. Latest consensus state: current_term: 1 leader_uuid: "e075aa3262494c72ae0fcd1d05734eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } } peers { permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } } peers { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } } }
I20260806 18:32:17.749691 27022 sys_catalog.cc:458] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [sys.catalog]: This master's current role is: LEADER
I20260806 18:32:17.751435 27045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:32:17.755929 27045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:32:17.761915 26817 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a [term 1 FOLLOWER]: Refusing update from remote peer e075aa3262494c72ae0fcd1d05734eb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:32:17.762766 27043 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:32:17.767125 26983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e075aa3262494c72ae0fcd1d05734eb0. Latest consensus state: current_term: 1 leader_uuid: "e075aa3262494c72ae0fcd1d05734eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } } peers { permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } } peers { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } } }
I20260806 18:32:17.767266 26983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:32:17.770989 26704 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d [term 1 FOLLOWER]: Refusing update from remote peer e075aa3262494c72ae0fcd1d05734eb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:32:17.771670 27043 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:32:17.776320 27006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e075aa3262494c72ae0fcd1d05734eb0. Latest consensus state: current_term: 1 leader_uuid: "e075aa3262494c72ae0fcd1d05734eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } } peers { permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } } peers { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } } }
I20260806 18:32:17.776467 27006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:32:17.777935 27043 sys_catalog.cc:455] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e075aa3262494c72ae0fcd1d05734eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } } peers { permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } } peers { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } } }
I20260806 18:32:17.778088 27043 sys_catalog.cc:458] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [sys.catalog]: This master's current role is: LEADER
I20260806 18:32:17.781965 27047 mvcc.cc:205] Tried to move back new op lower bound from 7315624500264448000 to 7315624500217860096. Current Snapshot: MvccSnapshot[applied={T|T < 7315624500264448000}]
I20260806 18:32:17.782737 26983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e075aa3262494c72ae0fcd1d05734eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } } peers { permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } } peers { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } } }
I20260806 18:32:17.782891 26983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:32:17.783501 27049 mvcc.cc:205] Tried to move back new op lower bound from 7315624500264448000 to 7315624500217860096. Current Snapshot: MvccSnapshot[applied={T|T < 7315624500264448000}]
I20260806 18:32:17.785507 27045 catalog_manager.cc:1347] Generated new cluster ID: 7eb36123bd974aa9b37bb7b397b90c81
I20260806 18:32:17.785578 27045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:32:17.802589 27045 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:32:17.805097 27006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e075aa3262494c72ae0fcd1d05734eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } } peers { permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } } peers { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } } }
I20260806 18:32:17.805236 27006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:32:17.803471 27043 sys_catalog.cc:455] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e075aa3262494c72ae0fcd1d05734eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c6a4c718a3b402d8010a861ed1ae42d" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34963 } } peers { permanent_uuid: "4f4821bd9f444fd799474b583f30c95a" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 34765 } } peers { permanent_uuid: "e075aa3262494c72ae0fcd1d05734eb0" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 45359 } } }
I20260806 18:32:17.805651 27043 sys_catalog.cc:458] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0 [sys.catalog]: This master's current role is: LEADER
I20260806 18:32:17.807047 27045 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:32:17.808710 27052 mvcc.cc:205] Tried to move back new op lower bound from 7315624500264448000 to 7315624500217860096. Current Snapshot: MvccSnapshot[applied={T|T < 7315624500264448000}]
I20260806 18:32:17.822660 27045 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e075aa3262494c72ae0fcd1d05734eb0: Generated new TSK 0
I20260806 18:32:17.823652 27045 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:32:18.705444 27020 catalog_manager.cc:1259] Loaded cluster ID: 7eb36123bd974aa9b37bb7b397b90c81
I20260806 18:32:18.705526 27020 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d: loading cluster ID for follower catalog manager: success
I20260806 18:32:18.707561 27020 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d: acquiring CA information for follower catalog manager: success
I20260806 18:32:18.708983 27020 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7c6a4c718a3b402d8010a861ed1ae42d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:32:18.709372 26999 catalog_manager.cc:1259] Loaded cluster ID: 7eb36123bd974aa9b37bb7b397b90c81
I20260806 18:32:18.709436 26999 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a: loading cluster ID for follower catalog manager: success
I20260806 18:32:18.711066 26999 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a: acquiring CA information for follower catalog manager: success
I20260806 18:32:18.712167 26999 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4f4821bd9f444fd799474b583f30c95a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:32:18.895533 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 26644
I20260806 18:32:18.917210 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 26757
I20260806 18:32:18.931384 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 26866
2026-08-06T18:32:18Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3169 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:32:18 dist-test-slave-5bn2 krb5kdc[27070](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:32:18 dist-test-slave-5bn2 krb5kdc[27070](info): set up 2 sockets
Aug 06 18:32:18 dist-test-slave-5bn2 krb5kdc[27070](info): commencing operation
krb5kdc: starting...
W20260806 18:32:20.055660 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:32:20 dist-test-slave-5bn2 krb5kdc[27070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:32:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:32:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:32:20.181924 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:37315
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:36203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:37315
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:32:20.298246 27088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:20.298556 27088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:20.298619 27088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:20.303277 27088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:32:20.303359 27088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:20.303385 27088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:20.303407 27088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:32:20.303436 27088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:32:20.309223 27088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:36203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:37315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:37315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:20.310405 27088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:20.311527 27088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:20.320079 27094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:20.320281 27097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:20.320079 27095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:20.320832 27088 server_base.cc:1034] running on GCE node
I20260806 18:32:20.321424 27088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:20.322561 27088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:20.323716 27088 hybrid_clock.cc:648] HybridClock initialized: now 1786041140323686 us; error 42 us; skew 500 ppm
Aug 06 18:32:20 dist-test-slave-5bn2 krb5kdc[27070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041140, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:20.329659 27088 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:32:20.330660 27088 webserver.cc:460] Webserver started at http://127.7.67.254:46309/ using document root <none> and password file <none>
I20260806 18:32:20.331264 27088 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:20.331331 27088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:20.331554 27088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:32:20.333658 27088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "fc3a05f5207440a2bdf77dd4bb749154"
format_stamp: "Formatted at 2026-08-06 18:32:20 on dist-test-slave-5bn2"
server_key: "53b52ae9afe464b66516c0fe8197ae3a"
server_key_iv: "53dcd682d699f82ef2f2dd99618f6bd0"
server_key_version: "encryptionkey@0"
I20260806 18:32:20.334221 27088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "fc3a05f5207440a2bdf77dd4bb749154"
format_stamp: "Formatted at 2026-08-06 18:32:20 on dist-test-slave-5bn2"
server_key: "53b52ae9afe464b66516c0fe8197ae3a"
server_key_iv: "53dcd682d699f82ef2f2dd99618f6bd0"
server_key_version: "encryptionkey@0"
I20260806 18:32:20.338044 27088 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:32:20.340592 27105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:20.341919 27088 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:32:20.342041 27088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "fc3a05f5207440a2bdf77dd4bb749154"
format_stamp: "Formatted at 2026-08-06 18:32:20 on dist-test-slave-5bn2"
server_key: "53b52ae9afe464b66516c0fe8197ae3a"
server_key_iv: "53dcd682d699f82ef2f2dd99618f6bd0"
server_key_version: "encryptionkey@0"
I20260806 18:32:20.342167 27088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:20.357345 27088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:20.361065 27088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:20.361282 27088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:20.368619 27088 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:37315
I20260806 18:32:20.368634 27180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:37315 every 8 connection(s)
I20260806 18:32:20.369634 27088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:32:20.372532 27182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:32:20.378314 27182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154: Bootstrap starting.
I20260806 18:32:20.379532 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 27088
I20260806 18:32:20.379678 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:32:20.380002 7439 external_mini_cluster.cc:1428] Setting key 799f00c385ce4e9c4f3cead4abbd8410
I20260806 18:32:20.381332 27182 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154: Neither blocks nor log segments found. Creating new log.
I20260806 18:32:20.382115 27182 log.cc:824] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:20.384114 27182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154: No bootstrap required, opened a new log
I20260806 18:32:20.387672 27182 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3a05f5207440a2bdf77dd4bb749154" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 37315 } }
I20260806 18:32:20.387910 27182 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:32:20.387964 27182 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3a05f5207440a2bdf77dd4bb749154, State: Initialized, Role: FOLLOWER
Aug 06 18:32:20 dist-test-slave-5bn2 krb5kdc[27070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:20.388479 27182 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3a05f5207440a2bdf77dd4bb749154" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 37315 } }
I20260806 18:32:20.388638 27182 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:32:20.388705 27182 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:32:20.388804 27182 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:32:20.389986 27182 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3a05f5207440a2bdf77dd4bb749154" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 37315 } }
I20260806 18:32:20.390348 27182 leader_election.cc:302] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc3a05f5207440a2bdf77dd4bb749154; no voters:
I20260806 18:32:20.390688 27182 leader_election.cc:288] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:32:20.390781 27189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:32:20.390986 27189 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154 [term 1 LEADER]: Becoming Leader. State: Replica: fc3a05f5207440a2bdf77dd4bb749154, State: Running, Role: LEADER
I20260806 18:32:20.391291 27189 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3a05f5207440a2bdf77dd4bb749154" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 37315 } }
I20260806 18:32:20.392087 27182 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:32:20.393405 27190 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc3a05f5207440a2bdf77dd4bb749154" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3a05f5207440a2bdf77dd4bb749154" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 37315 } } }
I20260806 18:32:20.393405 27191 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc3a05f5207440a2bdf77dd4bb749154. Latest consensus state: current_term: 1 leader_uuid: "fc3a05f5207440a2bdf77dd4bb749154" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3a05f5207440a2bdf77dd4bb749154" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 37315 } } }
I20260806 18:32:20.393556 27191 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154 [sys.catalog]: This master's current role is: LEADER
I20260806 18:32:20.393556 27190 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154 [sys.catalog]: This master's current role is: LEADER
I20260806 18:32:20.394065 27199 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:32:20.394663 27185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:20.381143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43156 (local address 127.7.67.254:37315)
0806 18:32:20.381617 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:20.381634 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:20.381670 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:32:20.382364 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:20.382384 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:20.382785 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:20.383186 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:20.384003 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:20.385008 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:20.385856 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:20.386346 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:20.386493 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:20.389174 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:20.389180 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:20.389197 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:20.389228 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:20.391033 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:20.391562 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:20.391568 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:20.391574 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:20.391694 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:20.391967 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:20.391971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:20.391973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:20.392477 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:20.392701 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:20.393360 (+ 659us) server_negotiation.cc:299] Negotiation successful
0806 18:32:20.393683 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":137,"threads_started":1}
I20260806 18:32:20.397425 27199 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:32:20.397718 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:32:20.403326 27199 catalog_manager.cc:1347] Generated new cluster ID: b50c4e91363d4691922cf7c51746dc35
I20260806 18:32:20.403396 27199 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:32:20.411777 27199 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:32:20.412730 27199 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:32:20.417359 27199 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc3a05f5207440a2bdf77dd4bb749154: Generated new TSK 0
I20260806 18:32:20.418051 27199 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:32:20.453648 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 27088
2026-08-06T18:32:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1519 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:32:20 dist-test-slave-5bn2 krb5kdc[27214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:32:20 dist-test-slave-5bn2 krb5kdc[27214](info): set up 2 sockets
Aug 06 18:32:20 dist-test-slave-5bn2 krb5kdc[27214](info): commencing operation
krb5kdc: starting...
W20260806 18:32:21.570592 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:32:21 dist-test-slave-5bn2 krb5kdc[27214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:32:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:32:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:32:21.695456 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34859
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:34859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:32:21.807394 27235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:21.807689 27235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:21.807729 27235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:21.811683 27235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:32:21.811744 27235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:21.811762 27235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:21.811779 27235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:32:21.811791 27235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:32:21.817082 27235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:34859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:21.818125 27235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:21.819125 27235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:21.827489 27241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:21.827488 27243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:21.827661 27235 server_base.cc:1034] running on GCE node
W20260806 18:32:21.827812 27240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:21.828390 27235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:21.829494 27235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:21.830664 27235 hybrid_clock.cc:648] HybridClock initialized: now 1786041141830647 us; error 28 us; skew 500 ppm
Aug 06 18:32:21 dist-test-slave-5bn2 krb5kdc[27214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041141, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:21.833525 27235 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:32:21.834442 27235 webserver.cc:460] Webserver started at http://127.7.67.254:39969/ using document root <none> and password file <none>
I20260806 18:32:21.835034 27235 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:21.835103 27235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:21.835296 27235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:32:21.837344 27235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "dea71849f1c249dc87990eb6fc53b1fe"
format_stamp: "Formatted at 2026-08-06 18:32:21 on dist-test-slave-5bn2"
server_key: "0fba15939327fbaa0ddf9d9f402cd4d4"
server_key_iv: "e9d66b549499ab3805843a3a4b598cf1"
server_key_version: "encryptionkey@0"
I20260806 18:32:21.837910 27235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "dea71849f1c249dc87990eb6fc53b1fe"
format_stamp: "Formatted at 2026-08-06 18:32:21 on dist-test-slave-5bn2"
server_key: "0fba15939327fbaa0ddf9d9f402cd4d4"
server_key_iv: "e9d66b549499ab3805843a3a4b598cf1"
server_key_version: "encryptionkey@0"
I20260806 18:32:21.841688 27235 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 18:32:21.844137 27252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:21.845368 27235 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:32:21.845490 27235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "dea71849f1c249dc87990eb6fc53b1fe"
format_stamp: "Formatted at 2026-08-06 18:32:21 on dist-test-slave-5bn2"
server_key: "0fba15939327fbaa0ddf9d9f402cd4d4"
server_key_iv: "e9d66b549499ab3805843a3a4b598cf1"
server_key_version: "encryptionkey@0"
I20260806 18:32:21.845592 27235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:21.863663 27235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:21.866930 27235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:21.867111 27235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:21.874085 27235 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:34859
I20260806 18:32:21.874344 27323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:34859 every 8 connection(s)
I20260806 18:32:21.875124 27235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:32:21.877974 27326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:32:21.882091 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 27235
I20260806 18:32:21.882237 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:32:21.882541 7439 external_mini_cluster.cc:1428] Setting key 25903fb9b90dd18027f5b7b56a06fefe
I20260806 18:32:21.884389 27326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe: Bootstrap starting.
I20260806 18:32:21.887413 27326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe: Neither blocks nor log segments found. Creating new log.
I20260806 18:32:21.888422 27326 log.cc:824] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:21.890399 27326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe: No bootstrap required, opened a new log
Aug 06 18:32:21 dist-test-slave-5bn2 krb5kdc[27214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:21.894320 27326 raft_consensus.cc:348] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea71849f1c249dc87990eb6fc53b1fe" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34859 } }
I20260806 18:32:21.894559 27326 raft_consensus.cc:374] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:32:21.894634 27326 raft_consensus.cc:729] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dea71849f1c249dc87990eb6fc53b1fe, State: Initialized, Role: FOLLOWER
I20260806 18:32:21.895161 27326 consensus_queue.cc:260] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea71849f1c249dc87990eb6fc53b1fe" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34859 } }
I20260806 18:32:21.895323 27326 raft_consensus.cc:388] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:32:21.895392 27326 raft_consensus.cc:482] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:32:21.895499 27326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:32:21.896445 27326 raft_consensus.cc:504] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea71849f1c249dc87990eb6fc53b1fe" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34859 } }
I20260806 18:32:21.896801 27326 leader_election.cc:302] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dea71849f1c249dc87990eb6fc53b1fe; no voters:
I20260806 18:32:21.897119 27326 leader_election.cc:288] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:32:21.897207 27336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:32:21.897383 27336 raft_consensus.cc:686] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 1 LEADER]: Becoming Leader. State: Replica: dea71849f1c249dc87990eb6fc53b1fe, State: Running, Role: LEADER
I20260806 18:32:21.897554 27332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:21.884256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36222 (local address 127.7.67.254:34859)
0806 18:32:21.884726 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:21.884745 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:21.884782 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:32:21.885634 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:21.885661 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:21.886116 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:21.886506 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:21.887397 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:21.888832 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:21.889687 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:21.890185 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:21.890342 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:21.892752 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:21.892757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:21.892771 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:21.892799 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:21.894410 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:21.894917 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:21.894923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:21.894929 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:21.895077 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:21.895348 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:21.895351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:21.895352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:21.895824 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:21.896022 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:21.896291 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 18:32:21.896605 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":127,"threads_started":1}
I20260806 18:32:21.897764 27336 consensus_queue.cc:237] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea71849f1c249dc87990eb6fc53b1fe" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34859 } }
I20260806 18:32:21.898460 27326 sys_catalog.cc:564] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:32:21.899471 27338 sys_catalog.cc:455] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader dea71849f1c249dc87990eb6fc53b1fe. Latest consensus state: current_term: 1 leader_uuid: "dea71849f1c249dc87990eb6fc53b1fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea71849f1c249dc87990eb6fc53b1fe" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34859 } } }
I20260806 18:32:21.899665 27338 sys_catalog.cc:458] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [sys.catalog]: This master's current role is: LEADER
I20260806 18:32:21.900025 27337 sys_catalog.cc:455] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dea71849f1c249dc87990eb6fc53b1fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea71849f1c249dc87990eb6fc53b1fe" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34859 } } }
I20260806 18:32:21.900180 27337 sys_catalog.cc:458] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [sys.catalog]: This master's current role is: LEADER
I20260806 18:32:21.900496 27345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:32:21.903183 27345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:32:21.908558 27345 catalog_manager.cc:1347] Generated new cluster ID: 07bb337fd64042d8b179ec83931e219c
I20260806 18:32:21.908629 27345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:32:21.931687 27345 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:32:21.932977 27345 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:32:21.944558 27345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe: Generated new TSK 0
I20260806 18:32:21.945276 27345 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
I20260806 18:32:21.981702 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:34859
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:32:22.096410 27362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:22.096683 27362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:22.096724 27362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:22.100862 27362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:22.100924 27362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:22.100986 27362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:32:22.106307 27362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:34859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:22.107344 27362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:22.108414 27362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:22.118130 27367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:22.118146 27370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:22.118244 27362 server_base.cc:1034] running on GCE node
W20260806 18:32:22.118175 27368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:22.119354 27362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:22.119999 27362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:22.121161 27362 hybrid_clock.cc:648] HybridClock initialized: now 1786041142121143 us; error 33 us; skew 500 ppm
Aug 06 18:32:22 dist-test-slave-5bn2 krb5kdc[27214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041142, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:22.123914 27362 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:32:22.124821 27362 webserver.cc:460] Webserver started at http://127.7.67.193:39407/ using document root <none> and password file <none>
I20260806 18:32:22.125365 27362 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:22.125450 27362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:22.125669 27362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:32:22.127799 27362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "a8fefa6195a948cfa4ed08c816f3e0e0"
format_stamp: "Formatted at 2026-08-06 18:32:22 on dist-test-slave-5bn2"
server_key: "e14b4e960fffcf9146f0ba9936757bb1"
server_key_iv: "d328765cd6835554a29b6bf5c6f6618d"
server_key_version: "encryptionkey@0"
I20260806 18:32:22.128387 27362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "a8fefa6195a948cfa4ed08c816f3e0e0"
format_stamp: "Formatted at 2026-08-06 18:32:22 on dist-test-slave-5bn2"
server_key: "e14b4e960fffcf9146f0ba9936757bb1"
server_key_iv: "d328765cd6835554a29b6bf5c6f6618d"
server_key_version: "encryptionkey@0"
I20260806 18:32:22.132138 27362 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:32:22.134549 27380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:22.135793 27362 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:32:22.135911 27362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "a8fefa6195a948cfa4ed08c816f3e0e0"
format_stamp: "Formatted at 2026-08-06 18:32:22 on dist-test-slave-5bn2"
server_key: "e14b4e960fffcf9146f0ba9936757bb1"
server_key_iv: "d328765cd6835554a29b6bf5c6f6618d"
server_key_version: "encryptionkey@0"
I20260806 18:32:22.136010 27362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:22.160962 27362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:22.163889 27362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:22.164078 27362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:22.164691 27362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:32:22.165556 27362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:32:22.165606 27362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:22.165663 27362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:32:22.165691 27362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:22.174306 27362 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:38173
I20260806 18:32:22.174337 27540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:38173 every 8 connection(s)
I20260806 18:32:22.175247 27362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:32:22.179005 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 27362
I20260806 18:32:22.179148 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:32:22.179469 7439 external_mini_cluster.cc:1428] Setting key cb6164bc25d5e5bb6cda90b31c5f519b
Aug 06 18:32:22 dist-test-slave-5bn2 krb5kdc[27214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041142, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:22.190234 27332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:22.177069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:44947 (local address 127.7.67.254:34859)
0806 18:32:22.177243 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:22.177248 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:22.178089 (+ 841us) server_negotiation.cc:407] Connection header received
0806 18:32:22.179628 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:22.179631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:22.179682 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:22.179756 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:22.181162 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.181708 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.183304 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.183653 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.183765 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:22.186361 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:22.186366 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:22.186370 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:22.186403 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:22.188106 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:22.188653 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:22.188657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:22.188659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:22.188754 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:22.189055 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:22.189058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:22.189060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:22.189295 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:22.189397 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:22.189897 (+ 500us) server_negotiation.cc:299] Negotiation successful
0806 18:32:22.190037 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
WARNING: no policy specified for kudu/127.7.67.194@KRBTEST.COM; defaulting to no policy
I20260806 18:32:22.191064 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:22.177467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:44947)
0806 18:32:22.177924 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:22.177972 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:22.179377 (+ 1405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:22.179943 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:22.179950 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:22.180386 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:22.181015 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:22.181036 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.181844 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.181847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:22.183156 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:22.183170 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.183801 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.183803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:22.183948 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:22.184531 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:22.184549 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:22.186162 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:22.188248 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:22.188254 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:22.188267 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:22.188550 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:22.188840 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:22.188842 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:22.188843 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:22.188975 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:22.189388 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:22.189393 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:22.189649 (+ 256us) client_negotiation.cc:769] Sending connection context
0806 18:32:22.189807 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 18:32:22.190089 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":114,"threads_started":1}
I20260806 18:32:22.192297 27541 heartbeater.cc:347] Connected to a master server at 127.7.67.254:34859
I20260806 18:32:22.192587 27541 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:22.193193 27541 heartbeater.cc:511] Master 127.7.67.254:34859 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.194@KRBTEST.COM" created.
I20260806 18:32:22.194818 27274 ts_manager.cc:194] Registered new tserver with Master: a8fefa6195a948cfa4ed08c816f3e0e0 (127.7.67.193:38173)
I20260806 18:32:22.195995 27274 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:44947
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
I20260806 18:32:22.209817 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:34859
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:32:22.326623 27546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:22.326906 27546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:22.326949 27546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:22.330920 27546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:22.330986 27546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:22.331043 27546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:32:22.336246 27546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:34859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:22.337291 27546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:22.338320 27546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:22.348474 27553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:22.348467 27552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:22.348474 27555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:22.348852 27546 server_base.cc:1034] running on GCE node
I20260806 18:32:22.349308 27546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:22.349994 27546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:22.351156 27546 hybrid_clock.cc:648] HybridClock initialized: now 1786041142351133 us; error 41 us; skew 500 ppm
Aug 06 18:32:22 dist-test-slave-5bn2 krb5kdc[27214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041142, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:22.353919 27546 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:32:22.354889 27546 webserver.cc:460] Webserver started at http://127.7.67.194:37153/ using document root <none> and password file <none>
I20260806 18:32:22.355455 27546 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:22.355545 27546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:22.355760 27546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:32:22.357861 27546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "d4f25ba1e34c4f6095d63a36a6c2babc"
format_stamp: "Formatted at 2026-08-06 18:32:22 on dist-test-slave-5bn2"
server_key: "b273328e413133af883fcc97c178280f"
server_key_iv: "1f97103bcda592e1871cb842aaa161d5"
server_key_version: "encryptionkey@0"
I20260806 18:32:22.358414 27546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "d4f25ba1e34c4f6095d63a36a6c2babc"
format_stamp: "Formatted at 2026-08-06 18:32:22 on dist-test-slave-5bn2"
server_key: "b273328e413133af883fcc97c178280f"
server_key_iv: "1f97103bcda592e1871cb842aaa161d5"
server_key_version: "encryptionkey@0"
I20260806 18:32:22.362205 27546 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:32:22.364836 27564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:22.366065 27546 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:32:22.366194 27546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "d4f25ba1e34c4f6095d63a36a6c2babc"
format_stamp: "Formatted at 2026-08-06 18:32:22 on dist-test-slave-5bn2"
server_key: "b273328e413133af883fcc97c178280f"
server_key_iv: "1f97103bcda592e1871cb842aaa161d5"
server_key_version: "encryptionkey@0"
I20260806 18:32:22.366297 27546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:22.379621 27546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:22.382436 27546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:22.382623 27546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:22.383255 27546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:32:22.384066 27546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:32:22.384114 27546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:22.384168 27546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:32:22.384194 27546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:22.392622 27546 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:43389
I20260806 18:32:22.392647 27726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:43389 every 8 connection(s)
I20260806 18:32:22.393510 27546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:32:22.396944 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 27546
I20260806 18:32:22.397135 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:32:22.397431 7439 external_mini_cluster.cc:1428] Setting key 985918a46b1b1985a215e6bdeb520225
Aug 06 18:32:22 dist-test-slave-5bn2 krb5kdc[27214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041142, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:22.408099 27332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:22.395280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:42079 (local address 127.7.67.254:34859)
0806 18:32:22.395432 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:22.395436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:22.396242 (+ 806us) server_negotiation.cc:407] Connection header received
0806 18:32:22.397830 (+ 1588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:22.397834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:22.397894 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:22.397986 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:22.399544 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.400207 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.401651 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.401977 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.402078 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:22.404538 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:22.404543 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:22.404546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:22.404574 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:22.405945 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:22.406530 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:22.406533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:22.406534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:22.406604 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:22.406993 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:22.406996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:22.406997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:22.407173 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:22.407246 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:22.407870 (+ 624us) server_negotiation.cc:299] Negotiation successful
0806 18:32:22.407974 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for kudu/127.7.67.195@KRBTEST.COM; defaulting to no policy
I20260806 18:32:22.409184 27732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:22.395652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:42079)
0806 18:32:22.396093 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:22.396136 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:22.397584 (+ 1448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:22.398172 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:22.398182 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:22.398579 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:22.399377 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:22.399398 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.400334 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.400338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:22.401530 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:22.401541 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.402121 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.402124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:22.402267 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:22.402868 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:22.402891 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:22.404355 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:22.406065 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:22.406073 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:22.406090 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:22.406439 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:22.406682 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:22.406685 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:22.406687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:22.406894 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:22.407275 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:22.407282 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:22.407621 (+ 339us) client_negotiation.cc:769] Sending connection context
0806 18:32:22.407843 (+ 222us) client_negotiation.cc:240] Negotiation successful
0806 18:32:22.408213 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":118,"threads_started":1}
I20260806 18:32:22.410372 27729 heartbeater.cc:347] Connected to a master server at 127.7.67.254:34859
I20260806 18:32:22.410641 27729 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.195@KRBTEST.COM" created.
I20260806 18:32:22.411252 27729 heartbeater.cc:511] Master 127.7.67.254:34859 requested a full tablet report, sending...
I20260806 18:32:22.412348 27274 ts_manager.cc:194] Registered new tserver with Master: d4f25ba1e34c4f6095d63a36a6c2babc (127.7.67.194:43389)
I20260806 18:32:22.412905 27274 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:42079
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
I20260806 18:32:22.425822 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:34859
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:32:22.542280 27735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:22.542575 27735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:22.542622 27735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:22.546882 27735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:22.546957 27735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:22.547017 27735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:32:22.552582 27735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:34859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:22.553637 27735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:22.554625 27735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:22.564338 27746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:22.564342 27742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:22.564342 27743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:22.564623 27735 server_base.cc:1034] running on GCE node
I20260806 18:32:22.565097 27735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:22.565752 27735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:22.566912 27735 hybrid_clock.cc:648] HybridClock initialized: now 1786041142566893 us; error 32 us; skew 500 ppm
Aug 06 18:32:22 dist-test-slave-5bn2 krb5kdc[27214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041142, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:22.569644 27735 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:32:22.570583 27735 webserver.cc:460] Webserver started at http://127.7.67.195:46515/ using document root <none> and password file <none>
I20260806 18:32:22.571193 27735 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:22.571265 27735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:22.571475 27735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:32:22.573534 27735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "d810a9bcacde44a59767edb24a41bf2d"
format_stamp: "Formatted at 2026-08-06 18:32:22 on dist-test-slave-5bn2"
server_key: "51e2effbbc6f921f03c930f512df7e1f"
server_key_iv: "bb3ed8436dbf018a7afb2ac1f14e0bd9"
server_key_version: "encryptionkey@0"
I20260806 18:32:22.574090 27735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "d810a9bcacde44a59767edb24a41bf2d"
format_stamp: "Formatted at 2026-08-06 18:32:22 on dist-test-slave-5bn2"
server_key: "51e2effbbc6f921f03c930f512df7e1f"
server_key_iv: "bb3ed8436dbf018a7afb2ac1f14e0bd9"
server_key_version: "encryptionkey@0"
I20260806 18:32:22.577872 27735 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 18:32:22.580348 27756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:22.581558 27735 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:32:22.581686 27735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "d810a9bcacde44a59767edb24a41bf2d"
format_stamp: "Formatted at 2026-08-06 18:32:22 on dist-test-slave-5bn2"
server_key: "51e2effbbc6f921f03c930f512df7e1f"
server_key_iv: "bb3ed8436dbf018a7afb2ac1f14e0bd9"
server_key_version: "encryptionkey@0"
I20260806 18:32:22.581782 27735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:22.599778 27735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:22.602995 27735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:22.603178 27735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:22.603767 27735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:32:22.604616 27735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:32:22.604667 27735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:22.604723 27735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:32:22.604750 27735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:22.614068 27735 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:40461
I20260806 18:32:22.614094 27928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:40461 every 8 connection(s)
I20260806 18:32:22.614996 27735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:32:22.623277 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 27735
I20260806 18:32:22.623436 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:32:22.623703 7439 external_mini_cluster.cc:1428] Setting key 7bc8c5d19645b83529e31adf38f55435
Aug 06 18:32:22 dist-test-slave-5bn2 krb5kdc[27214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041142, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:22.628360 27332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:22.616829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:54983 (local address 127.7.67.254:34859)
0806 18:32:22.616979 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:22.616983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:22.617880 (+ 897us) server_negotiation.cc:407] Connection header received
0806 18:32:22.619213 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:22.619216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:22.619275 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:22.619348 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:22.620527 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.621052 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.622169 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.622385 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.622480 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:22.625010 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:22.625014 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:22.625017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:22.625045 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:22.626403 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:22.626924 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:22.626927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:22.626928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:22.627010 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:22.627308 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:22.627311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:22.627312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:22.627503 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:22.627578 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:22.628092 (+ 514us) server_negotiation.cc:299] Negotiation successful
0806 18:32:22.628209 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:32:22.629060 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:22.617191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:54983)
0806 18:32:22.617700 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:22.617757 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:22.618995 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:22.619533 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:22.619540 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:22.619908 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:22.620383 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:22.620399 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.621171 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.621174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:22.622052 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:22.622061 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.622535 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.622540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:22.622704 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:22.623352 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:22.623368 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:22.624819 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:22.626510 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:22.626514 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:22.626525 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:22.626780 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:22.627083 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:22.627085 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:22.627086 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:22.627222 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:22.627578 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:22.627582 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:22.627818 (+ 236us) client_negotiation.cc:769] Sending connection context
0806 18:32:22.627977 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 18:32:22.628230 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":133,"threads_started":1}
I20260806 18:32:22.630225 27929 heartbeater.cc:347] Connected to a master server at 127.7.67.254:34859
I20260806 18:32:22.630450 27929 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:22.630955 27929 heartbeater.cc:511] Master 127.7.67.254:34859 requested a full tablet report, sending...
I20260806 18:32:22.631956 27274 ts_manager.cc:194] Registered new tserver with Master: d810a9bcacde44a59767edb24a41bf2d (127.7.67.195:40461)
I20260806 18:32:22.632519 27274 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:54983
I20260806 18:32:22.637600 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:32:22.646780 27332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:22.639669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36230 (local address 127.7.67.254:34859)
0806 18:32:22.639812 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:22.639816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:22.640034 (+ 218us) server_negotiation.cc:407] Connection header received
0806 18:32:22.640116 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:22.640118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:22.640163 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:22.640233 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:22.641018 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.641543 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.642369 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.642571 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.642633 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:22.643924 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:22.643928 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:22.643930 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:22.643952 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:22.645199 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:22.645615 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:22.645618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:22.645620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:22.645697 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:22.645944 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:22.645947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:22.645949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:22.646164 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:22.646281 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:22.646515 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 18:32:22.646625 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 18:32:22.649773 27274 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:32:22.652887 27274 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:32:22.667297 27952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:22.662321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:43389 (local address 127.0.0.1:40778)
0806 18:32:22.662921 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:22.662936 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:22.663025 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:22.663566 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:22.663569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:22.663588 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:22.663686 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:22.663692 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.664737 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.664741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:22.665893 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:22.665904 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.666714 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.666718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:22.666820 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:22.666898 (+ 78us) client_negotiation.cc:769] Sending connection context
0806 18:32:22.667010 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 18:32:22.667141 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":101,"threads_started":1}
I20260806 18:32:22.667547 27953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:22.662760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40778 (local address 127.7.67.194:43389)
0806 18:32:22.663163 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:22.663174 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:22.663195 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:32:22.663268 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:22.663273 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:22.663438 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:22.663578 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:22.663806 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.664592 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.666026 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.666599 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.666696 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:22.666879 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:22.667052 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 18:32:22.667401 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":184,"threads_started":1}
I20260806 18:32:22.667624 27950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:22.661736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:40461 (local address 127.0.0.1:34240)
0806 18:32:22.662580 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:22.662595 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:22.662699 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:22.663744 (+ 1045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:22.663747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:22.663765 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:22.663837 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:22.663842 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.665160 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.665163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:22.666020 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:22.666031 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.667202 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.667206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:22.667295 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:22.667337 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 18:32:22.667389 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 18:32:22.667468 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"mutex_wait_us":23,"thread_start_us":122,"threads_started":1}
I20260806 18:32:22.668298 27951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:22.662116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34240 (local address 127.7.67.195:40461)
0806 18:32:22.663269 (+ 1153us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:22.663278 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:22.663303 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:32:22.663390 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:22.663396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:22.663580 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:22.663770 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:22.663934 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.665016 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.666165 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.667051 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.667830 (+ 779us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:22.667991 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:22.668048 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 18:32:22.668142 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1058,"thread_start_us":164,"threads_started":1}
I20260806 18:32:22.668737 27946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:22.661571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:38173 (local address 127.0.0.1:33902)
0806 18:32:22.662370 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:22.662445 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:22.662572 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:22.663380 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:22.663386 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:22.663413 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:22.663581 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:22.663589 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.665133 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.665137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:22.667182 (+ 2045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:22.667191 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.668275 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.668278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:22.668372 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:22.668483 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 18:32:22.668554 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 18:32:22.668622 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":78,"threads_started":1}
I20260806 18:32:22.668874 27949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:22.661709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33902 (local address 127.7.67.193:38173)
0806 18:32:22.662841 (+ 1132us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:22.662885 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:22.662911 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:32:22.662988 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:22.662994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:22.663203 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:22.663359 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:22.663741 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.664964 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.667322 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.668156 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.668272 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:22.668421 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:22.668614 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 18:32:22.668719 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1035,"thread_start_us":116,"threads_started":1}
I20260806 18:32:22.671201 27833 tablet_service.cc:1467] Processing CreateTablet for tablet 33cd1f31f41e4c709779fd8553331660 (DEFAULT_TABLE table=test-table [id=8aad3c7da167462f8ee85b7f91883a61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:32:22.671231 27635 tablet_service.cc:1467] Processing CreateTablet for tablet 33cd1f31f41e4c709779fd8553331660 (DEFAULT_TABLE table=test-table [id=8aad3c7da167462f8ee85b7f91883a61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:32:22.671587 27443 tablet_service.cc:1467] Processing CreateTablet for tablet 33cd1f31f41e4c709779fd8553331660 (DEFAULT_TABLE table=test-table [id=8aad3c7da167462f8ee85b7f91883a61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:32:22.672309 27833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33cd1f31f41e4c709779fd8553331660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:32:22.672312 27635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33cd1f31f41e4c709779fd8553331660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:32:22.672595 27443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33cd1f31f41e4c709779fd8553331660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:32:22.679126 27954 tablet_bootstrap.cc:492] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d: Bootstrap starting.
I20260806 18:32:22.680091 27955 tablet_bootstrap.cc:492] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc: Bootstrap starting.
I20260806 18:32:22.680192 27956 tablet_bootstrap.cc:492] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0: Bootstrap starting.
I20260806 18:32:22.682104 27954 tablet_bootstrap.cc:654] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d: Neither blocks nor log segments found. Creating new log.
I20260806 18:32:22.682377 27956 tablet_bootstrap.cc:654] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0: Neither blocks nor log segments found. Creating new log.
I20260806 18:32:22.682520 27955 tablet_bootstrap.cc:654] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc: Neither blocks nor log segments found. Creating new log.
I20260806 18:32:22.683120 27954 log.cc:824] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:22.683233 27955 log.cc:824] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:22.683588 27956 log.cc:824] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:22.685549 27955 tablet_bootstrap.cc:492] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc: No bootstrap required, opened a new log
I20260806 18:32:22.685597 27954 tablet_bootstrap.cc:492] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d: No bootstrap required, opened a new log
I20260806 18:32:22.685595 27956 tablet_bootstrap.cc:492] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0: No bootstrap required, opened a new log
I20260806 18:32:22.685745 27955 ts_tablet_manager.cc:1397] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:32:22.685778 27956 ts_tablet_manager.cc:1397] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:32:22.685778 27954 ts_tablet_manager.cc:1397] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260806 18:32:22.688699 27955 raft_consensus.cc:348] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } } peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } }
I20260806 18:32:22.688930 27955 raft_consensus.cc:374] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:32:22.689013 27955 raft_consensus.cc:729] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4f25ba1e34c4f6095d63a36a6c2babc, State: Initialized, Role: FOLLOWER
I20260806 18:32:22.689441 27955 consensus_queue.cc:260] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } } peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } }
I20260806 18:32:22.689668 27956 raft_consensus.cc:348] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } } peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } }
I20260806 18:32:22.689668 27954 raft_consensus.cc:348] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } } peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } }
I20260806 18:32:22.689940 27954 raft_consensus.cc:374] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:32:22.689972 27956 raft_consensus.cc:374] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:32:22.690022 27956 raft_consensus.cc:729] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8fefa6195a948cfa4ed08c816f3e0e0, State: Initialized, Role: FOLLOWER
I20260806 18:32:22.690112 27729 heartbeater.cc:503] Master 127.7.67.254:34859 was elected leader, sending a full tablet report...
I20260806 18:32:22.690121 27954 raft_consensus.cc:729] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d810a9bcacde44a59767edb24a41bf2d, State: Initialized, Role: FOLLOWER
I20260806 18:32:22.690446 27955 ts_tablet_manager.cc:1428] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:32:22.690526 27954 consensus_queue.cc:260] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } } peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } }
I20260806 18:32:22.690549 27956 consensus_queue.cc:260] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } } peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } }
I20260806 18:32:22.691598 27954 ts_tablet_manager.cc:1428] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260806 18:32:22.691885 27956 ts_tablet_manager.cc:1428] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:32:22.691974 27929 heartbeater.cc:503] Master 127.7.67.254:34859 was elected leader, sending a full tablet report...
I20260806 18:32:22.692256 27541 heartbeater.cc:503] Master 127.7.67.254:34859 was elected leader, sending a full tablet report...
I20260806 18:32:22.847016 27973 raft_consensus.cc:482] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:32:22.847252 27973 raft_consensus.cc:504] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } } peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } }
I20260806 18:32:22.849231 27973 leader_election.cc:288] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d810a9bcacde44a59767edb24a41bf2d (127.7.67.195:40461), d4f25ba1e34c4f6095d63a36a6c2babc (127.7.67.194:43389)
I20260806 18:32:22.852936 27951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:22.848990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:50165 (local address 127.7.67.195:40461)
0806 18:32:22.849125 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:22.849130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:22.849282 (+ 152us) server_negotiation.cc:407] Connection header received
0806 18:32:22.849500 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:22.849504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:22.849566 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:22.849658 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:22.850132 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.850731 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.851910 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.852360 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.852462 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:22.852509 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:22.852728 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 18:32:22.852801 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:32:22.852965 27975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:22.849444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:43389 (local address 127.7.67.193:47959)
0806 18:32:22.849824 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:22.849836 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:22.849929 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:22.850204 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:22.850206 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:22.850223 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:22.850321 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:22.850325 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.851136 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.851139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:22.851923 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:22.851934 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.852563 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.852566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:22.852647 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:22.852666 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:32:22.852738 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 18:32:22.852804 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":110,"threads_started":1}
I20260806 18:32:22.852952 27953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:22.849570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:47959 (local address 127.7.67.194:43389)
0806 18:32:22.849714 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:22.849719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:22.849926 (+ 207us) server_negotiation.cc:407] Connection header received
0806 18:32:22.850040 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:22.850043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:22.850112 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:22.850187 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:22.850437 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.851028 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.852047 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.852438 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.852548 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:22.852599 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:22.852768 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 18:32:22.852841 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 18:32:22.852974 27974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:22.848838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:40461 (local address 127.7.67.193:50165)
0806 18:32:22.849163 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:22.849180 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:22.849372 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:22.849776 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:22.849780 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:22.849803 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:22.849933 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:22.849939 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.850835 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.850838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:22.851773 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:22.851784 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.852499 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.852503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:22.852599 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:22.852624 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:32:22.852684 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 18:32:22.852755 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":2,"thread_start_us":119,"threads_started":1}
I20260806 18:32:22.853809 27665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33cd1f31f41e4c709779fd8553331660" candidate_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" is_pre_election: true
I20260806 18:32:22.853807 27866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33cd1f31f41e4c709779fd8553331660" candidate_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d810a9bcacde44a59767edb24a41bf2d" is_pre_election: true
I20260806 18:32:22.854154 27665 raft_consensus.cc:2454] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8fefa6195a948cfa4ed08c816f3e0e0 in term 0.
I20260806 18:32:22.854154 27866 raft_consensus.cc:2454] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8fefa6195a948cfa4ed08c816f3e0e0 in term 0.
I20260806 18:32:22.854635 27382 leader_election.cc:302] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8fefa6195a948cfa4ed08c816f3e0e0, d810a9bcacde44a59767edb24a41bf2d; no voters:
I20260806 18:32:22.854912 27973 raft_consensus.cc:2781] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:32:22.855000 27973 raft_consensus.cc:482] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:32:22.855051 27973 raft_consensus.cc:3037] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:32:22.856105 27973 raft_consensus.cc:504] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } } peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } }
I20260806 18:32:22.856423 27973 leader_election.cc:288] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 1 election: Requested vote from peers d810a9bcacde44a59767edb24a41bf2d (127.7.67.195:40461), d4f25ba1e34c4f6095d63a36a6c2babc (127.7.67.194:43389)
I20260806 18:32:22.856801 27866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33cd1f31f41e4c709779fd8553331660" candidate_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d810a9bcacde44a59767edb24a41bf2d"
I20260806 18:32:22.856840 27665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33cd1f31f41e4c709779fd8553331660" candidate_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc"
I20260806 18:32:22.856945 27665 raft_consensus.cc:3037] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:32:22.856940 27866 raft_consensus.cc:3037] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:32:22.858244 27866 raft_consensus.cc:2454] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8fefa6195a948cfa4ed08c816f3e0e0 in term 1.
I20260806 18:32:22.858244 27665 raft_consensus.cc:2454] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8fefa6195a948cfa4ed08c816f3e0e0 in term 1.
I20260806 18:32:22.858525 27382 leader_election.cc:302] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8fefa6195a948cfa4ed08c816f3e0e0, d810a9bcacde44a59767edb24a41bf2d; no voters:
I20260806 18:32:22.858683 27973 raft_consensus.cc:2781] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:32:22.858927 27973 raft_consensus.cc:686] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 LEADER]: Becoming Leader. State: Replica: a8fefa6195a948cfa4ed08c816f3e0e0, State: Running, Role: LEADER
I20260806 18:32:22.859251 27973 consensus_queue.cc:237] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } } peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } }
I20260806 18:32:22.862560 27274 catalog_manager.cc:5495] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a8fefa6195a948cfa4ed08c816f3e0e0 (127.7.67.193). New cstate: current_term: 1 leader_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } health_report { overall_health: UNKNOWN } } }
W20260806 18:32:22.865918 27930 tablet.cc:2369] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:32:22.894654 27730 tablet.cc:2369] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:32:22.902683 27949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:22.899386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33910 (local address 127.7.67.193:38173)
0806 18:32:22.899525 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:22.899528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:22.899543 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:32:22.899654 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:22.899656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:22.899715 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:22.899800 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:32:22.900036 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.900549 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.901320 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:22.901528 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:22.901593 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:22.901827 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:32:22.902255 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:32:22.902345 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:22.902477 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 18:32:22.902532 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:32:22.909112 27665 raft_consensus.cc:1261] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc [term 1 FOLLOWER]: Refusing update from remote peer a8fefa6195a948cfa4ed08c816f3e0e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:32:22.909113 27866 raft_consensus.cc:1261] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d [term 1 FOLLOWER]: Refusing update from remote peer a8fefa6195a948cfa4ed08c816f3e0e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:32:22.909843 27973 consensus_queue.cc:1035] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:32:22.910056 27976 consensus_queue.cc:1035] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:32:22.918067 27985 mvcc.cc:205] Tried to move back new op lower bound from 7315624521347547136 to 7315624521154105344. Current Snapshot: MvccSnapshot[applied={T|T < 7315624521347547136}]
I20260806 18:32:22.919664 27984 mvcc.cc:205] Tried to move back new op lower bound from 7315624521347547136 to 7315624521154105344. Current Snapshot: MvccSnapshot[applied={T|T < 7315624521347547136}]
I20260806 18:32:22.926518 27273 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:32:22.926729 27273 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:32:22.929342 27273 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe: Sending DeleteTablet for 3 replicas of tablet 33cd1f31f41e4c709779fd8553331660
I20260806 18:32:22.930223 27443 tablet_service.cc:1514] Processing DeleteTablet for tablet 33cd1f31f41e4c709779fd8553331660 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:32:22 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:33902
I20260806 18:32:22.930370 27635 tablet_service.cc:1514] Processing DeleteTablet for tablet 33cd1f31f41e4c709779fd8553331660 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:32:22 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:40778
I20260806 18:32:22.930542 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 27362
I20260806 18:32:22.930814 27833 tablet_service.cc:1514] Processing DeleteTablet for tablet 33cd1f31f41e4c709779fd8553331660 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:32:22 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:34240
I20260806 18:32:22.934183 28000 tablet_replica.cc:330] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d: stopping tablet replica
I20260806 18:32:22.934542 28000 raft_consensus.cc:2229] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:32:22.934870 28000 raft_consensus.cc:2258] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:32:22.938879 27999 tablet_replica.cc:330] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc: stopping tablet replica
I20260806 18:32:22.938879 28000 ts_tablet_manager.cc:1905] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:32:22.940047 27999 raft_consensus.cc:2229] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:32:22.940511 27999 raft_consensus.cc:2258] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:32:22.941690 28000 ts_tablet_manager.cc:1918] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:32:22.941814 28000 log.cc:1196] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/33cd1f31f41e4c709779fd8553331660
I20260806 18:32:22.942101 28000 ts_tablet_manager.cc:1939] T 33cd1f31f41e4c709779fd8553331660 P d810a9bcacde44a59767edb24a41bf2d: Deleting consensus metadata
I20260806 18:32:22.943619 27255 catalog_manager.cc:4841] TS d810a9bcacde44a59767edb24a41bf2d (127.7.67.195:40461): tablet 33cd1f31f41e4c709779fd8553331660 (table test-table [id=8aad3c7da167462f8ee85b7f91883a61]) successfully deleted
I20260806 18:32:22.943743 27999 ts_tablet_manager.cc:1905] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:32:22.945957 27999 ts_tablet_manager.cc:1918] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:32:22.946048 27999 log.cc:1196] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/33cd1f31f41e4c709779fd8553331660
I20260806 18:32:22.946300 27999 ts_tablet_manager.cc:1939] T 33cd1f31f41e4c709779fd8553331660 P d4f25ba1e34c4f6095d63a36a6c2babc: Deleting consensus metadata
I20260806 18:32:22.947526 27255 catalog_manager.cc:4841] TS d4f25ba1e34c4f6095d63a36a6c2babc (127.7.67.194:43389): tablet 33cd1f31f41e4c709779fd8553331660 (table test-table [id=8aad3c7da167462f8ee85b7f91883a61]) successfully deleted
W20260806 18:32:22.948719 27257 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.193:38173 (error 108)
I20260806 18:32:22.949546 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 27546
I20260806 18:32:22.949808 27946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:22.949509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:38173 (local address 127.0.0.1:33924)
0806 18:32:22.949642 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:22.949719 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 18:32:22.951874 27257 catalog_manager.cc:4584] TS a8fefa6195a948cfa4ed08c816f3e0e0 (127.7.67.193:38173): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 33cd1f31f41e4c709779fd8553331660: Network error: Client connection negotiation failed: client connection to 127.7.67.193:38173: connect: Connection refused (error 111)
I20260806 18:32:22.963835 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 27735
I20260806 18:32:22.971441 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 27235
I20260806 18:32:22.984591 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34859
--webserver_interface=127.7.67.254
--webserver_port=39969
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:34859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:32:23.097626 28003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:23.097945 28003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:23.098002 28003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:23.102031 28003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:32:23.102102 28003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:23.102128 28003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:23.102149 28003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:32:23.102169 28003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:32:23.107978 28003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:34859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=39969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:23.109123 28003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:23.110183 28003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:23.124230 28010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:23.124231 28011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:23.124373 28003 server_base.cc:1034] running on GCE node
W20260806 18:32:23.124231 28014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:23.125190 28003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:23.126446 28003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:23.127642 28003 hybrid_clock.cc:648] HybridClock initialized: now 1786041143127622 us; error 33 us; skew 500 ppm
Aug 06 18:32:23 dist-test-slave-5bn2 krb5kdc[27214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:23.130709 28003 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:32:23.131767 28003 webserver.cc:460] Webserver started at http://127.7.67.254:39969/ using document root <none> and password file <none>
I20260806 18:32:23.132409 28003 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:23.132508 28003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:23.136785 28003 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:32:23.139221 28021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:23.140537 28003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:32:23.140677 28003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "dea71849f1c249dc87990eb6fc53b1fe"
format_stamp: "Formatted at 2026-08-06 18:32:21 on dist-test-slave-5bn2"
server_key: "0fba15939327fbaa0ddf9d9f402cd4d4"
server_key_iv: "e9d66b549499ab3805843a3a4b598cf1"
server_key_version: "encryptionkey@0"
I20260806 18:32:23.141331 28003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:23.157338 28003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:23.160429 28003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:23.160647 28003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:23.168690 28003 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:34859
I20260806 18:32:23.169037 28093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:34859 every 8 connection(s)
I20260806 18:32:23.169950 28003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:32:23.171340 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 28003
I20260806 18:32:23.171797 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:38173
--local_ip_for_outbound_sockets=127.7.67.193
--tserver_master_addrs=127.7.67.254:34859
--webserver_port=39407
--webserver_interface=127.7.67.193
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:32:23.175575 28094 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:32:23.177423 28094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe: Bootstrap starting.
I20260806 18:32:23.195747 28094 log.cc:824] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:23.204777 28094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:32:23.205163 28094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe: Bootstrap complete.
I20260806 18:32:23.209383 28094 raft_consensus.cc:348] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea71849f1c249dc87990eb6fc53b1fe" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34859 } }
I20260806 18:32:23.209810 28094 raft_consensus.cc:729] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dea71849f1c249dc87990eb6fc53b1fe, State: Initialized, Role: FOLLOWER
I20260806 18:32:23.210330 28094 consensus_queue.cc:260] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea71849f1c249dc87990eb6fc53b1fe" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34859 } }
I20260806 18:32:23.210464 28094 raft_consensus.cc:388] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:32:23.210541 28094 raft_consensus.cc:482] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:32:23.210645 28094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:32:23.212613 28094 raft_consensus.cc:504] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea71849f1c249dc87990eb6fc53b1fe" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34859 } }
I20260806 18:32:23.212960 28094 leader_election.cc:302] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dea71849f1c249dc87990eb6fc53b1fe; no voters:
I20260806 18:32:23.213300 28094 leader_election.cc:288] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:32:23.213428 28101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:32:23.213613 28101 raft_consensus.cc:686] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 2 LEADER]: Becoming Leader. State: Replica: dea71849f1c249dc87990eb6fc53b1fe, State: Running, Role: LEADER
I20260806 18:32:23.213917 28101 consensus_queue.cc:237] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea71849f1c249dc87990eb6fc53b1fe" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34859 } }
I20260806 18:32:23.214643 28094 sys_catalog.cc:564] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:32:23.214964 28104 sys_catalog.cc:455] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader dea71849f1c249dc87990eb6fc53b1fe. Latest consensus state: current_term: 2 leader_uuid: "dea71849f1c249dc87990eb6fc53b1fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea71849f1c249dc87990eb6fc53b1fe" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34859 } } }
I20260806 18:32:23.215111 28104 sys_catalog.cc:458] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [sys.catalog]: This master's current role is: LEADER
I20260806 18:32:23.215363 28103 sys_catalog.cc:455] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dea71849f1c249dc87990eb6fc53b1fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea71849f1c249dc87990eb6fc53b1fe" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34859 } } }
I20260806 18:32:23.215485 28103 sys_catalog.cc:458] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [sys.catalog]: This master's current role is: LEADER
I20260806 18:32:23.215937 28107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:32:23.221033 28107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:32:23.221690 28107 catalog_manager.cc:1259] Loaded cluster ID: 07bb337fd64042d8b179ec83931e219c
I20260806 18:32:23.221758 28107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:32:23.223237 28107 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 18:32:23.227977 28107 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 18:32:23.234064 28107 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:32:23.235006 28107 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe: Loaded TSK: 0
I20260806 18:32:23.236263 28107 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:32:23.294955 28097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:23.295246 28097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:23.295325 28097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:23.300061 28097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:23.300138 28097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:23.300222 28097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:32:23.305655 28097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:38173
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=39407
--tserver_master_addrs=127.7.67.254:34859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:23.306797 28097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:23.307922 28097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:23.318945 28138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:23.318955 28140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:23.318955 28143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:23.319139 28097 server_base.cc:1034] running on GCE node
I20260806 18:32:23.319718 28097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:23.320446 28097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:23.321619 28097 hybrid_clock.cc:648] HybridClock initialized: now 1786041143321594 us; error 42 us; skew 500 ppm
Aug 06 18:32:23 dist-test-slave-5bn2 krb5kdc[27214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:23.324467 28097 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:32:23.325417 28097 webserver.cc:460] Webserver started at http://127.7.67.193:39407/ using document root <none> and password file <none>
I20260806 18:32:23.326020 28097 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:23.326110 28097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:23.330282 28097 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:32:23.332629 28151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:23.334034 28097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:32:23.334151 28097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "a8fefa6195a948cfa4ed08c816f3e0e0"
format_stamp: "Formatted at 2026-08-06 18:32:22 on dist-test-slave-5bn2"
server_key: "e14b4e960fffcf9146f0ba9936757bb1"
server_key_iv: "d328765cd6835554a29b6bf5c6f6618d"
server_key_version: "encryptionkey@0"
I20260806 18:32:23.334702 28097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:23.356194 28097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:23.359287 28097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:23.359472 28097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:23.360106 28097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:32:23.361241 28162 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:32:23.365439 28097 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:32:23.365501 28097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260806 18:32:23.365553 28097 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:32:23.367295 28097 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:32:23.367357 28097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260806 18:32:23.367699 28162 tablet_bootstrap.cc:492] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0: Bootstrap starting.
I20260806 18:32:23.377983 28097 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:38173
I20260806 18:32:23.378003 28309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:38173 every 8 connection(s)
I20260806 18:32:23.379271 28097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:32:23.380589 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 28097
I20260806 18:32:23.381033 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:43389
--local_ip_for_outbound_sockets=127.7.67.194
--tserver_master_addrs=127.7.67.254:34859
--webserver_port=37153
--webserver_interface=127.7.67.194
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:32:23.389125 28162 log.cc:824] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:32:23 dist-test-slave-5bn2 krb5kdc[27214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:23.398123 28312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:23.381482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:55937 (local address 127.7.67.254:34859)
0806 18:32:23.381887 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:23.381906 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:23.382369 (+ 463us) server_negotiation.cc:407] Connection header received
0806 18:32:23.383731 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:23.383746 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:23.384151 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:23.384493 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:23.386305 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.387880 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.389591 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.390261 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.390440 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:23.392929 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:23.392935 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:23.392948 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:23.392976 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:23.394553 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:23.395126 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:23.395131 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:23.395135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:23.395261 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:23.395616 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:23.395619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:23.395620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:23.396114 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:23.396260 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:23.397057 (+ 797us) server_negotiation.cc:299] Negotiation successful
0806 18:32:23.397339 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":125,"threads_started":1}
I20260806 18:32:23.398145 28313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:23.381410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:55937)
0806 18:32:23.382185 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:23.382240 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:23.383170 (+ 930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:23.384958 (+ 1788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:23.384966 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:23.385452 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:23.386075 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:23.386099 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.388067 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.388071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:23.389448 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:23.389461 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.390417 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.390421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:23.390609 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:23.391200 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:23.391217 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:23.392726 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:23.394674 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:23.394680 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:23.394693 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:23.395015 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:23.395354 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:23.395357 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:23.395359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:23.395515 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:23.396327 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:23.396334 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:23.396645 (+ 311us) client_negotiation.cc:769] Sending connection context
0806 18:32:23.396838 (+ 193us) client_negotiation.cc:240] Negotiation successful
0806 18:32:23.397201 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":135,"threads_started":1}
I20260806 18:32:23.398252 28162 tablet_bootstrap.cc:492] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:32:23.398658 28162 tablet_bootstrap.cc:492] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0: Bootstrap complete.
I20260806 18:32:23.400425 28162 ts_tablet_manager.cc:1397] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0: Time spent bootstrapping tablet: real 0.033s user 0.023s sys 0.008s
I20260806 18:32:23.400841 28310 heartbeater.cc:347] Connected to a master server at 127.7.67.254:34859
I20260806 18:32:23.401079 28310 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:23.401557 28310 heartbeater.cc:511] Master 127.7.67.254:34859 requested a full tablet report, sending...
I20260806 18:32:23.403860 28039 ts_manager.cc:194] Registered new tserver with Master: a8fefa6195a948cfa4ed08c816f3e0e0 (127.7.67.193:38173)
I20260806 18:32:23.403465 28162 raft_consensus.cc:348] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } } peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } }
I20260806 18:32:23.403961 28162 raft_consensus.cc:729] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8fefa6195a948cfa4ed08c816f3e0e0, State: Initialized, Role: FOLLOWER
I20260806 18:32:23.404469 28162 consensus_queue.cc:260] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } } peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } }
I20260806 18:32:23.405472 28162 ts_tablet_manager.cc:1428] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:32:23.407696 28039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:55937
I20260806 18:32:23.410307 28310 heartbeater.cc:503] Master 127.7.67.254:34859 was elected leader, sending a full tablet report...
I20260806 18:32:23.413549 28320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:23.406883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33926 (local address 127.7.67.193:38173)
0806 18:32:23.407175 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:23.407187 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:23.408851 (+ 1664us) server_negotiation.cc:407] Connection header received
0806 18:32:23.408907 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:23.408911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:23.409445 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:23.409585 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:23.409836 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.410628 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.412253 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.412941 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.413029 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:23.413125 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:23.413356 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 18:32:23.413441 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":126,"threads_started":1}
I20260806 18:32:23.413602 28319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:23.406809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:38173 (local address 127.0.0.1:33926)
0806 18:32:23.408564 (+ 1755us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:23.408613 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:23.408747 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:23.409594 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:23.409599 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:23.409622 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:23.409734 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:23.409740 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.410753 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.410756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:23.412108 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:23.412121 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.413071 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.413074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:23.413188 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:23.413227 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 18:32:23.413336 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 18:32:23.413461 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1554,"thread_start_us":144,"threads_started":1}
I20260806 18:32:23.414563 28216 tablet_service.cc:1514] Processing DeleteTablet for tablet 33cd1f31f41e4c709779fd8553331660 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:32:22 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:33926
I20260806 18:32:23.415585 28321 tablet_replica.cc:330] stopping tablet replica
I20260806 18:32:23.415865 28321 raft_consensus.cc:2229] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:32:23.416095 28321 raft_consensus.cc:2258] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:32:23.417050 28321 ts_tablet_manager.cc:1905] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:32:23.420588 28321 ts_tablet_manager.cc:1918] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:32:23.420674 28321 log.cc:1196] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/wal/wals/33cd1f31f41e4c709779fd8553331660
I20260806 18:32:23.420907 28321 ts_tablet_manager.cc:1939] T 33cd1f31f41e4c709779fd8553331660 P a8fefa6195a948cfa4ed08c816f3e0e0: Deleting consensus metadata
I20260806 18:32:23.422528 28026 catalog_manager.cc:4841] TS a8fefa6195a948cfa4ed08c816f3e0e0 (127.7.67.193:38173): tablet 33cd1f31f41e4c709779fd8553331660 (table test-table [id=8aad3c7da167462f8ee85b7f91883a61]) successfully deleted
W20260806 18:32:23.504587 28315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:23.504879 28315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:23.504942 28315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:23.509027 28315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:23.509100 28315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:23.509182 28315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:32:23.514784 28315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:43389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=37153
--tserver_master_addrs=127.7.67.254:34859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:23.515929 28315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:23.516981 28315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:23.526778 28331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:23.526780 28330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:23.526780 28335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:23.527153 28315 server_base.cc:1034] running on GCE node
I20260806 18:32:23.527598 28315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:23.528265 28315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:23.529433 28315 hybrid_clock.cc:648] HybridClock initialized: now 1786041143529415 us; error 33 us; skew 500 ppm
Aug 06 18:32:23 dist-test-slave-5bn2 krb5kdc[27214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:23.532855 28315 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:32:23.533860 28315 webserver.cc:460] Webserver started at http://127.7.67.194:37153/ using document root <none> and password file <none>
I20260806 18:32:23.534431 28315 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:23.534533 28315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:23.538605 28315 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:32:23.540773 28348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:23.542088 28315 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:32:23.542208 28315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "d4f25ba1e34c4f6095d63a36a6c2babc"
format_stamp: "Formatted at 2026-08-06 18:32:22 on dist-test-slave-5bn2"
server_key: "b273328e413133af883fcc97c178280f"
server_key_iv: "1f97103bcda592e1871cb842aaa161d5"
server_key_version: "encryptionkey@0"
I20260806 18:32:23.542780 28315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:23.572423 28315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:23.575678 28315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:23.575876 28315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:23.576508 28315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:32:23.577383 28315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:32:23.577435 28315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:23.577491 28315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:32:23.577520 28315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:23.586818 28315 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:43389
I20260806 18:32:23.586897 28522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:43389 every 8 connection(s)
I20260806 18:32:23.587911 28315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:32:23.590148 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 28315
I20260806 18:32:23.590664 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:40461
--local_ip_for_outbound_sockets=127.7.67.195
--tserver_master_addrs=127.7.67.254:34859
--webserver_port=46515
--webserver_interface=127.7.67.195
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:32:23 dist-test-slave-5bn2 krb5kdc[27214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:23.602435 28312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:23.589808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:43459 (local address 127.7.67.254:34859)
0806 18:32:23.589970 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:23.589974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:23.591168 (+ 1194us) server_negotiation.cc:407] Connection header received
0806 18:32:23.592548 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:23.592552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:23.592624 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:23.592700 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:23.594021 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.594760 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.596023 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.596267 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.596363 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:23.598978 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:23.598982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:23.598984 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:23.599012 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:23.600334 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:23.600848 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:23.600851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:23.600852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:23.600942 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:23.601276 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:23.601278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:23.601280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:23.601470 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:23.601557 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:23.602105 (+ 548us) server_negotiation.cc:299] Negotiation successful
0806 18:32:23.602255 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 18:32:23.603339 28525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:23.590353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:43459)
0806 18:32:23.590992 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:23.591044 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:23.592292 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:23.592927 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:23.592935 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:23.593320 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:23.593839 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:23.593864 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.594952 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.594956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:23.595883 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:23.595895 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.596461 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.596468 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:23.596632 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:23.597244 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:23.597261 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:23.598757 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:23.600454 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:23.600461 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:23.600478 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:23.600740 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:23.601026 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:23.601029 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:23.601031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:23.601177 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:23.601572 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:23.601579 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:23.601841 (+ 262us) client_negotiation.cc:769] Sending connection context
0806 18:32:23.602022 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 18:32:23.602377 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":163,"threads_started":1}
I20260806 18:32:23.604650 28523 heartbeater.cc:347] Connected to a master server at 127.7.67.254:34859
I20260806 18:32:23.604888 28523 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:23.605399 28523 heartbeater.cc:511] Master 127.7.67.254:34859 requested a full tablet report, sending...
I20260806 18:32:23.606527 28039 ts_manager.cc:194] Registered new tserver with Master: d4f25ba1e34c4f6095d63a36a6c2babc (127.7.67.194:43389)
I20260806 18:32:23.607321 28039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:43459
W20260806 18:32:23.707222 28526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:23.707479 28526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:23.707520 28526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:23.712638 28526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:23.712728 28526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:23.712810 28526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:32:23.720562 28526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:40461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=46515
--tserver_master_addrs=127.7.67.254:34859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:23.723430 28526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:23.724738 28526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:23.734510 28537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:23.734597 28539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:23.734556 28536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:23.734565 28526 server_base.cc:1034] running on GCE node
I20260806 18:32:23.735272 28526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:23.735898 28526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:23.737087 28526 hybrid_clock.cc:648] HybridClock initialized: now 1786041143737054 us; error 42 us; skew 500 ppm
Aug 06 18:32:23 dist-test-slave-5bn2 krb5kdc[27214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:23.739912 28526 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:32:23.740777 28526 webserver.cc:460] Webserver started at http://127.7.67.195:46515/ using document root <none> and password file <none>
I20260806 18:32:23.741315 28526 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:23.741385 28526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:23.745427 28526 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 18:32:23.747768 28549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:23.749110 28526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:32:23.749233 28526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "d810a9bcacde44a59767edb24a41bf2d"
format_stamp: "Formatted at 2026-08-06 18:32:22 on dist-test-slave-5bn2"
server_key: "51e2effbbc6f921f03c930f512df7e1f"
server_key_iv: "bb3ed8436dbf018a7afb2ac1f14e0bd9"
server_key_version: "encryptionkey@0"
I20260806 18:32:23.749794 28526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:23.763404 28526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:23.766474 28526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:23.766628 28526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:23.767252 28526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:32:23.768072 28526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:32:23.768119 28526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:23.768159 28526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:32:23.768179 28526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:23.776767 28526 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:40461
I20260806 18:32:23.776804 28717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:40461 every 8 connection(s)
I20260806 18:32:23.777806 28526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:32:23.779170 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 28526
Aug 06 18:32:23 dist-test-slave-5bn2 krb5kdc[27214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:23.787634 28724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:23.779882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36244 (local address 127.7.67.254:34859)
0806 18:32:23.780134 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:23.780138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:23.780226 (+ 88us) server_negotiation.cc:407] Connection header received
0806 18:32:23.780390 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:23.780394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:23.780445 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:23.780519 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:23.781325 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.781867 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.782814 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.783115 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.783206 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:23.784620 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:23.784625 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:23.784628 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:23.784652 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:23.786013 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:23.786419 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:23.786423 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:23.786424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:23.786509 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:23.786763 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:23.786766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:23.786768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:23.786982 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:23.787186 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:23.787331 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 18:32:23.787450 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":94,"threads_started":1}
I20260806 18:32:23.791000 28312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:23.779637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:50237 (local address 127.7.67.254:34859)
0806 18:32:23.779761 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:23.779764 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:23.780754 (+ 990us) server_negotiation.cc:407] Connection header received
0806 18:32:23.782080 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:23.782083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:23.782133 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:23.782207 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:23.783382 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.784055 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.785278 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.785579 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.785681 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:23.788064 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:23.788068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:23.788071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:23.788096 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:23.789340 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:23.789789 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:23.789792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:23.789793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:23.789863 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:23.790135 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:23.790137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:23.790138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:23.790280 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:23.790347 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:23.790760 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 18:32:23.790845 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:32:23.791868 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:23.780129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:50237)
0806 18:32:23.780601 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:23.780655 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:23.781893 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:23.782359 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:23.782366 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:23.782731 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:23.783278 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:23.783292 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.784183 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.784186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:23.785163 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:23.785174 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.785732 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.785736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:23.785909 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:23.786377 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:23.786393 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:23.787890 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:23.789440 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:23.789446 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:23.789460 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:23.789709 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:23.789936 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:23.789938 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:23.789939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:23.790058 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:23.790348 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:23.790353 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:23.790577 (+ 224us) client_negotiation.cc:769] Sending connection context
0806 18:32:23.790728 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 18:32:23.791039 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":105,"threads_started":1}
I20260806 18:32:23.792956 28719 heartbeater.cc:347] Connected to a master server at 127.7.67.254:34859
I20260806 18:32:23.793190 28719 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:23.793704 28719 heartbeater.cc:511] Master 127.7.67.254:34859 requested a full tablet report, sending...
I20260806 18:32:23.794693 28039 ts_manager.cc:194] Registered new tserver with Master: d810a9bcacde44a59767edb24a41bf2d (127.7.67.195:40461)
I20260806 18:32:23.795259 28039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:50237
I20260806 18:32:23.800997 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:32:23.804857 28312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:23.801584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36252 (local address 127.7.67.254:34859)
0806 18:32:23.801717 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:23.801721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:23.801895 (+ 174us) server_negotiation.cc:407] Connection header received
0806 18:32:23.802028 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:23.802030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:23.802127 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:23.802205 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:32:23.802375 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.802944 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.803705 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.803900 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.803975 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:23.804200 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:32:23.804571 (+ 371us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:32:23.804654 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:23.804682 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 18:32:23.804727 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:32:23.805485 28039 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:32:23.808837 28039 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:32:23.816715 28216 tablet_service.cc:1467] Processing CreateTablet for tablet 5444f1c9d7e14a8bbaa9068f614bcce2 (DEFAULT_TABLE table=test-table [id=9c69d60e46124a56abc7fa6227f0f8a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:32:23.817438 28216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5444f1c9d7e14a8bbaa9068f614bcce2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:32:23.820276 28162 tablet_bootstrap.cc:492] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0: Bootstrap starting.
I20260806 18:32:23.821079 28162 tablet_bootstrap.cc:654] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0: Neither blocks nor log segments found. Creating new log.
I20260806 18:32:23.821928 28319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:23.815601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:40461 (local address 127.0.0.1:34246)
0806 18:32:23.815710 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:23.815726 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:23.815802 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:23.816931 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:23.816934 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:23.816953 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:23.817060 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:23.817066 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.818173 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.818176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:23.818973 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:23.818984 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.821502 (+ 2518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.821505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:23.821610 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:23.821635 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:32:23.821716 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 18:32:23.821799 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 18:32:23.822139 28162 tablet_bootstrap.cc:492] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0: No bootstrap required, opened a new log
I20260806 18:32:23.822201 28737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:23.815932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34246 (local address 127.7.67.195:40461)
0806 18:32:23.816486 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:23.816500 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:23.816525 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:32:23.816606 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:23.816612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:23.816799 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:23.816956 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:23.817153 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.818053 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.819111 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.820095 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.820206 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:23.820461 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:23.821932 (+ 1471us) server_negotiation.cc:299] Negotiation successful
0806 18:32:23.822039 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":163,"threads_started":1}
I20260806 18:32:23.822220 28162 ts_tablet_manager.cc:1397] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:32:23.822978 28162 raft_consensus.cc:348] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } } peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } }
I20260806 18:32:23.823107 28162 raft_consensus.cc:374] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:32:23.823156 28162 raft_consensus.cc:729] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8fefa6195a948cfa4ed08c816f3e0e0, State: Initialized, Role: FOLLOWER
I20260806 18:32:23.823284 28162 consensus_queue.cc:260] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } } peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } }
I20260806 18:32:23.823558 28162 ts_tablet_manager.cc:1428] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 18:32:23.823699 28735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:23.815910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:43389 (local address 127.0.0.1:40794)
0806 18:32:23.816337 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:23.816350 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:23.816435 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:23.817330 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:23.817333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:23.817352 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:23.817440 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:23.817446 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.819001 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.819004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:23.820095 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:23.820105 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.823285 (+ 3180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.823289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:23.823381 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:23.823405 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:32:23.823504 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 18:32:23.823574 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":107,"threads_started":1}
I20260806 18:32:23.824182 28738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:23.816287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40794 (local address 127.7.67.194:43389)
0806 18:32:23.816720 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:23.816733 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:23.816755 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:32:23.816831 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:23.816836 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:23.817174 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:23.817584 (+ 410us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:23.817751 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.818791 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.822102 (+ 3311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.822964 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.823136 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:23.823290 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:23.823925 (+ 635us) server_negotiation.cc:299] Negotiation successful
0806 18:32:23.824029 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":172,"threads_started":1}
I20260806 18:32:23.825316 28618 tablet_service.cc:1467] Processing CreateTablet for tablet 5444f1c9d7e14a8bbaa9068f614bcce2 (DEFAULT_TABLE table=test-table [id=9c69d60e46124a56abc7fa6227f0f8a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:32:23.826347 28618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5444f1c9d7e14a8bbaa9068f614bcce2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:32:23.826478 28418 tablet_service.cc:1467] Processing CreateTablet for tablet 5444f1c9d7e14a8bbaa9068f614bcce2 (DEFAULT_TABLE table=test-table [id=9c69d60e46124a56abc7fa6227f0f8a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:32:23.827521 28418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5444f1c9d7e14a8bbaa9068f614bcce2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:32:23.833359 28740 tablet_bootstrap.cc:492] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d: Bootstrap starting.
I20260806 18:32:23.833545 28741 tablet_bootstrap.cc:492] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc: Bootstrap starting.
I20260806 18:32:23.835875 28741 tablet_bootstrap.cc:654] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc: Neither blocks nor log segments found. Creating new log.
I20260806 18:32:23.835894 28740 tablet_bootstrap.cc:654] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d: Neither blocks nor log segments found. Creating new log.
I20260806 18:32:23.836632 28741 log.cc:824] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:23.836632 28740 log.cc:824] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:23.838475 28740 tablet_bootstrap.cc:492] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d: No bootstrap required, opened a new log
I20260806 18:32:23.838469 28741 tablet_bootstrap.cc:492] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc: No bootstrap required, opened a new log
I20260806 18:32:23.838667 28741 ts_tablet_manager.cc:1397] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:32:23.838688 28740 ts_tablet_manager.cc:1397] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:32:23.841606 28741 raft_consensus.cc:348] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } } peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } }
I20260806 18:32:23.841610 28740 raft_consensus.cc:348] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } } peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } }
I20260806 18:32:23.841872 28741 raft_consensus.cc:374] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:32:23.841881 28740 raft_consensus.cc:374] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:32:23.841933 28741 raft_consensus.cc:729] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4f25ba1e34c4f6095d63a36a6c2babc, State: Initialized, Role: FOLLOWER
I20260806 18:32:23.841934 28740 raft_consensus.cc:729] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d810a9bcacde44a59767edb24a41bf2d, State: Initialized, Role: FOLLOWER
I20260806 18:32:23.842343 28740 consensus_queue.cc:260] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } } peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } }
I20260806 18:32:23.842343 28741 consensus_queue.cc:260] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } } peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } }
I20260806 18:32:23.843060 28523 heartbeater.cc:503] Master 127.7.67.254:34859 was elected leader, sending a full tablet report...
I20260806 18:32:23.843282 28740 ts_tablet_manager.cc:1428] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260806 18:32:23.843286 28741 ts_tablet_manager.cc:1428] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:32:23.843621 28719 heartbeater.cc:503] Master 127.7.67.254:34859 was elected leader, sending a full tablet report...
W20260806 18:32:23.880791 28311 tablet.cc:2369] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:32:23.959057 28746 raft_consensus.cc:482] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:32:23.959300 28746 raft_consensus.cc:504] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } } peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } }
I20260806 18:32:23.960517 28746 leader_election.cc:288] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d810a9bcacde44a59767edb24a41bf2d (127.7.67.195:40461), a8fefa6195a948cfa4ed08c816f3e0e0 (127.7.67.193:38173)
I20260806 18:32:23.965185 28737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:23.960988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:45791 (local address 127.7.67.195:40461)
0806 18:32:23.961109 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:23.961114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:23.961130 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:32:23.961193 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:23.961197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:23.961247 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:23.961326 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:23.961668 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.962460 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.963968 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.964603 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.964726 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:23.964785 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:23.964974 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 18:32:23.965047 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:32:23.965188 28749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:23.960886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:53037 (local address 127.7.67.193:38173)
0806 18:32:23.961176 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:23.961180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:23.961391 (+ 211us) server_negotiation.cc:407] Connection header received
0806 18:32:23.961505 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:23.961507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:23.961547 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:23.961621 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:23.961872 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.962673 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.964072 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.964694 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.964797 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:23.964844 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:23.964978 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 18:32:23.965047 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":87,"threads_started":1}
I20260806 18:32:23.965237 28525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:23.960809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:40461 (local address 127.7.67.194:45791)
0806 18:32:23.960922 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:23.960937 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:23.961039 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:23.961373 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:23.961376 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:23.961417 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:23.961554 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:23.961561 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.962577 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.962580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:23.963828 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:23.963840 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.964722 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.964724 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:23.964838 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:23.964867 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:32:23.964947 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 18:32:23.965067 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":23,"spinlock_wait_cycles":30336}
I20260806 18:32:23.965247 28750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:23.960809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:38173 (local address 127.7.67.194:53037)
0806 18:32:23.961291 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:23.961303 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:23.961424 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:23.961653 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:23.961656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:23.961674 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:23.961771 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:23.961775 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.962790 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.962793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:23.963947 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:23.963956 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:23.964791 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:23.964794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:23.964879 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:23.964898 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:32:23.964961 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 18:32:23.965067 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":146,"spinlock_wait_cycles":8704,"thread_start_us":117,"threads_started":1}
I20260806 18:32:23.966022 28647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5444f1c9d7e14a8bbaa9068f614bcce2" candidate_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d810a9bcacde44a59767edb24a41bf2d" is_pre_election: true
I20260806 18:32:23.966004 28244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5444f1c9d7e14a8bbaa9068f614bcce2" candidate_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" is_pre_election: true
I20260806 18:32:23.966326 28647 raft_consensus.cc:2454] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4f25ba1e34c4f6095d63a36a6c2babc in term 0.
I20260806 18:32:23.966326 28244 raft_consensus.cc:2454] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4f25ba1e34c4f6095d63a36a6c2babc in term 0.
I20260806 18:32:23.966817 28354 leader_election.cc:302] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8fefa6195a948cfa4ed08c816f3e0e0, d4f25ba1e34c4f6095d63a36a6c2babc; no voters:
I20260806 18:32:23.967131 28746 raft_consensus.cc:2781] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:32:23.967212 28746 raft_consensus.cc:482] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:32:23.967262 28746 raft_consensus.cc:3037] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:32:23.968317 28746 raft_consensus.cc:504] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } } peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } }
I20260806 18:32:23.968675 28746 leader_election.cc:288] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [CANDIDATE]: Term 1 election: Requested vote from peers d810a9bcacde44a59767edb24a41bf2d (127.7.67.195:40461), a8fefa6195a948cfa4ed08c816f3e0e0 (127.7.67.193:38173)
I20260806 18:32:23.969048 28647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5444f1c9d7e14a8bbaa9068f614bcce2" candidate_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d810a9bcacde44a59767edb24a41bf2d"
I20260806 18:32:23.969051 28244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5444f1c9d7e14a8bbaa9068f614bcce2" candidate_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0"
I20260806 18:32:23.969178 28244 raft_consensus.cc:3037] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:32:23.969173 28647 raft_consensus.cc:3037] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:32:23.970285 28244 raft_consensus.cc:2454] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4f25ba1e34c4f6095d63a36a6c2babc in term 1.
I20260806 18:32:23.970300 28647 raft_consensus.cc:2454] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4f25ba1e34c4f6095d63a36a6c2babc in term 1.
I20260806 18:32:23.970579 28352 leader_election.cc:302] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4f25ba1e34c4f6095d63a36a6c2babc, d810a9bcacde44a59767edb24a41bf2d; no voters:
I20260806 18:32:23.970747 28746 raft_consensus.cc:2781] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:32:23.971011 28746 raft_consensus.cc:686] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [term 1 LEADER]: Becoming Leader. State: Replica: d4f25ba1e34c4f6095d63a36a6c2babc, State: Running, Role: LEADER
I20260806 18:32:23.971361 28746 consensus_queue.cc:237] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } } peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } }
I20260806 18:32:23.974463 28039 catalog_manager.cc:5495] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc reported cstate change: term changed from 0 to 1, leader changed from <none> to d4f25ba1e34c4f6095d63a36a6c2babc (127.7.67.194). New cstate: current_term: 1 leader_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260806 18:32:24.015630 28738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:24.012144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40800 (local address 127.7.67.194:43389)
0806 18:32:24.012284 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:24.012289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:24.012304 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:32:24.012415 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:24.012417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:24.012480 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:24.012579 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:32:24.012744 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:24.013262 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:24.014059 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:24.014262 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:24.014333 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:24.014656 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:32:24.015198 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:32:24.015298 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:24.015404 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 18:32:24.015472 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:32:24.021587 28647 raft_consensus.cc:1261] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d [term 1 FOLLOWER]: Refusing update from remote peer d4f25ba1e34c4f6095d63a36a6c2babc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:32:24.021584 28244 raft_consensus.cc:1261] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Refusing update from remote peer d4f25ba1e34c4f6095d63a36a6c2babc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:32:24.022308 28751 consensus_queue.cc:1035] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:32:24.022497 28746 consensus_queue.cc:1035] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:32:24.028779 28722 tablet.cc:2369] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:32:24.032509 28759 mvcc.cc:205] Tried to move back new op lower bound from 7315624525905276928 to 7315624525709238272. Current Snapshot: MvccSnapshot[applied={T|T < 7315624525905276928 or (T in {7315624525905276928})}]
I20260806 18:32:24.040143 28039 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:32:24.040334 28039 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:32:24.042904 28039 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe: Sending DeleteTablet for 3 replicas of tablet 5444f1c9d7e14a8bbaa9068f614bcce2
I20260806 18:32:24.043650 28216 tablet_service.cc:1514] Processing DeleteTablet for tablet 5444f1c9d7e14a8bbaa9068f614bcce2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:32:24 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:33926
I20260806 18:32:24.043905 28418 tablet_service.cc:1514] Processing DeleteTablet for tablet 5444f1c9d7e14a8bbaa9068f614bcce2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:32:24 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:40794
I20260806 18:32:24.043906 28618 tablet_service.cc:1514] Processing DeleteTablet for tablet 5444f1c9d7e14a8bbaa9068f614bcce2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:32:24 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:34246
I20260806 18:32:24.044210 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 28097
I20260806 18:32:24.055624 28771 tablet_replica.cc:330] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d: stopping tablet replica
I20260806 18:32:24.056598 28771 raft_consensus.cc:2229] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:32:24.057081 28771 raft_consensus.cc:2258] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:32:24.057677 28772 tablet_replica.cc:330] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc: stopping tablet replica
I20260806 18:32:24.058066 28772 raft_consensus.cc:2229] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:32:24.058589 28772 raft_consensus.cc:2258] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:32:24.059067 28771 ts_tablet_manager.cc:1905] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:32:24.060501 28772 ts_tablet_manager.cc:1905] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:32:24.062050 28771 ts_tablet_manager.cc:1918] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:32:24.062170 28771 log.cc:1196] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/5444f1c9d7e14a8bbaa9068f614bcce2
I20260806 18:32:24.062520 28771 ts_tablet_manager.cc:1939] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d810a9bcacde44a59767edb24a41bf2d: Deleting consensus metadata
I20260806 18:32:24.063609 28772 ts_tablet_manager.cc:1918] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:32:24.063781 28772 log.cc:1196] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/5444f1c9d7e14a8bbaa9068f614bcce2
I20260806 18:32:24.064069 28024 catalog_manager.cc:4841] TS d810a9bcacde44a59767edb24a41bf2d (127.7.67.195:40461): tablet 5444f1c9d7e14a8bbaa9068f614bcce2 (table test-table [id=9c69d60e46124a56abc7fa6227f0f8a8]) successfully deleted
I20260806 18:32:24.064162 28772 ts_tablet_manager.cc:1939] T 5444f1c9d7e14a8bbaa9068f614bcce2 P d4f25ba1e34c4f6095d63a36a6c2babc: Deleting consensus metadata
I20260806 18:32:24.065326 28024 catalog_manager.cc:4841] TS d4f25ba1e34c4f6095d63a36a6c2babc (127.7.67.194:43389): tablet 5444f1c9d7e14a8bbaa9068f614bcce2 (table test-table [id=9c69d60e46124a56abc7fa6227f0f8a8]) successfully deleted
W20260806 18:32:24.065706 28026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.193:38173 (error 108)
I20260806 18:32:24.066332 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 28315
I20260806 18:32:24.070103 28735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:24.066493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:38173 (local address 127.0.0.1:33932)
0806 18:32:24.069841 (+ 3348us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:24.069971 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3225}
W20260806 18:32:24.073011 28026 catalog_manager.cc:4584] TS a8fefa6195a948cfa4ed08c816f3e0e0 (127.7.67.193:38173): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5444f1c9d7e14a8bbaa9068f614bcce2: Network error: Client connection negotiation failed: client connection to 127.7.67.193:38173: connect: Connection refused (error 111)
I20260806 18:32:24.086320 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 28526
I20260806 18:32:24.097018 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 28003
I20260806 18:32:24.112207 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34859
--webserver_interface=127.7.67.254
--webserver_port=39969
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:34859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:32:24.225011 28774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:24.225288 28774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:24.225344 28774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:24.229324 28774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:32:24.229389 28774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:24.229416 28774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:24.229437 28774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:32:24.229460 28774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:32:24.234764 28774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:34859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=39969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:24.235857 28774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:24.236923 28774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:24.245424 28783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:24.245409 28784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:24.245404 28787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:24.245534 28774 server_base.cc:1034] running on GCE node
I20260806 18:32:24.246279 28774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:24.247438 28774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:24.248613 28774 hybrid_clock.cc:648] HybridClock initialized: now 1786041144248594 us; error 32 us; skew 500 ppm
Aug 06 18:32:24 dist-test-slave-5bn2 krb5kdc[27214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041144, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:24.251394 28774 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:32:24.252339 28774 webserver.cc:460] Webserver started at http://127.7.67.254:39969/ using document root <none> and password file <none>
I20260806 18:32:24.252898 28774 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:24.252969 28774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:24.256879 28774 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:32:24.259163 28796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:24.260443 28774 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:32:24.260572 28774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "dea71849f1c249dc87990eb6fc53b1fe"
format_stamp: "Formatted at 2026-08-06 18:32:21 on dist-test-slave-5bn2"
server_key: "0fba15939327fbaa0ddf9d9f402cd4d4"
server_key_iv: "e9d66b549499ab3805843a3a4b598cf1"
server_key_version: "encryptionkey@0"
I20260806 18:32:24.261155 28774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:24.276023 28774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:24.278841 28774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:24.279075 28774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:24.286247 28774 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:34859
I20260806 18:32:24.286546 28877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:34859 every 8 connection(s)
I20260806 18:32:24.287439 28774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:32:24.288982 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 28774
I20260806 18:32:24.289502 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:38173
--local_ip_for_outbound_sockets=127.7.67.193
--tserver_master_addrs=127.7.67.254:34859
--webserver_port=39407
--webserver_interface=127.7.67.193
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:32:24.292855 28878 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:32:24.295176 28878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe: Bootstrap starting.
I20260806 18:32:24.319898 28878 log.cc:824] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:24.333875 28878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:32:24.334359 28878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe: Bootstrap complete.
I20260806 18:32:24.338615 28878 raft_consensus.cc:348] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea71849f1c249dc87990eb6fc53b1fe" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34859 } }
I20260806 18:32:24.339098 28878 raft_consensus.cc:729] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: dea71849f1c249dc87990eb6fc53b1fe, State: Initialized, Role: FOLLOWER
I20260806 18:32:24.339648 28878 consensus_queue.cc:260] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea71849f1c249dc87990eb6fc53b1fe" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34859 } }
I20260806 18:32:24.339771 28878 raft_consensus.cc:388] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:32:24.339836 28878 raft_consensus.cc:482] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:32:24.339932 28878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 2 FOLLOWER]: Advancing to term 3
I20260806 18:32:24.341939 28878 raft_consensus.cc:504] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea71849f1c249dc87990eb6fc53b1fe" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34859 } }
I20260806 18:32:24.342273 28878 leader_election.cc:302] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dea71849f1c249dc87990eb6fc53b1fe; no voters:
I20260806 18:32:24.342597 28878 leader_election.cc:288] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 18:32:24.342742 28884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 3 FOLLOWER]: Leader election won for term 3
I20260806 18:32:24.343020 28884 raft_consensus.cc:686] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [term 3 LEADER]: Becoming Leader. State: Replica: dea71849f1c249dc87990eb6fc53b1fe, State: Running, Role: LEADER
I20260806 18:32:24.343343 28884 consensus_queue.cc:237] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea71849f1c249dc87990eb6fc53b1fe" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34859 } }
I20260806 18:32:24.343864 28878 sys_catalog.cc:564] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:32:24.344818 28885 sys_catalog.cc:455] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "dea71849f1c249dc87990eb6fc53b1fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea71849f1c249dc87990eb6fc53b1fe" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34859 } } }
I20260806 18:32:24.344947 28885 sys_catalog.cc:458] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [sys.catalog]: This master's current role is: LEADER
I20260806 18:32:24.345225 28886 sys_catalog.cc:455] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader dea71849f1c249dc87990eb6fc53b1fe. Latest consensus state: current_term: 3 leader_uuid: "dea71849f1c249dc87990eb6fc53b1fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea71849f1c249dc87990eb6fc53b1fe" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34859 } } }
I20260806 18:32:24.345304 28886 sys_catalog.cc:458] T 00000000000000000000000000000000 P dea71849f1c249dc87990eb6fc53b1fe [sys.catalog]: This master's current role is: LEADER
I20260806 18:32:24.345937 28898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:32:24.351840 28898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:32:24.352504 28898 catalog_manager.cc:1259] Loaded cluster ID: 07bb337fd64042d8b179ec83931e219c
I20260806 18:32:24.352566 28898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 18:32:24.353652 28898 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb3a620f9ad google::LogMessage::Fail() at ??:0
@ 0x7fb3a621288a google::LogMessage::SendToLog() at ??:0
@ 0x7fb3a620f4de google::LogMessage::Flush() at ??:0
@ 0x7fb3a6210a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb3b0b4efdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fb3b0b4fd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb3b0b4c579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fb3b0b90640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fb3a84613d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb3a8461c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fb3a84633f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fb3a8452512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb3a9a966db start_thread at ??:0
@ 0x7fb3a4d0d61f clone at ??:0
W20260806 18:32:24.436035 28882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:24.436264 28882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:24.436305 28882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:24.440728 28882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:24.440783 28882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:24.440843 28882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:32:24.446270 28882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:38173
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=39407
--tserver_master_addrs=127.7.67.254:34859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:24.447443 28882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:24.448484 28882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:24.458173 28943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:24.458400 28945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:24.458170 28948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:24.458665 28882 server_base.cc:1034] running on GCE node
I20260806 18:32:24.459128 28882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:24.459777 28882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:24.460973 28882 hybrid_clock.cc:648] HybridClock initialized: now 1786041144460952 us; error 36 us; skew 500 ppm
Aug 06 18:32:24 dist-test-slave-5bn2 krb5kdc[27214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041144, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:24.464053 28882 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:32:24.465016 28882 webserver.cc:460] Webserver started at http://127.7.67.193:39407/ using document root <none> and password file <none>
I20260806 18:32:24.465607 28882 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:24.465678 28882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:24.469820 28882 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260806 18:32:24.472292 28960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:24.473743 28882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:32:24.473866 28882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "a8fefa6195a948cfa4ed08c816f3e0e0"
format_stamp: "Formatted at 2026-08-06 18:32:22 on dist-test-slave-5bn2"
server_key: "e14b4e960fffcf9146f0ba9936757bb1"
server_key_iv: "d328765cd6835554a29b6bf5c6f6618d"
server_key_version: "encryptionkey@0"
I20260806 18:32:24.474447 28882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:24.503311 28882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:24.506639 28882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:24.506836 28882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:24.507511 28882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:32:24.508764 28970 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:32:24.514609 28882 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:32:24.514688 28882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.001s
I20260806 18:32:24.514755 28882 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:32:24.516520 28882 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:32:24.516572 28882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 18:32:24.516919 28970 tablet_bootstrap.cc:492] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0: Bootstrap starting.
I20260806 18:32:24.526572 28882 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:38173
I20260806 18:32:24.526885 29120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:38173 every 8 connection(s)
I20260806 18:32:24.527696 28882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:32:24.528560 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 28882
I20260806 18:32:24.528959 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:43389
--local_ip_for_outbound_sockets=127.7.67.194
--tserver_master_addrs=127.7.67.254:34859
--webserver_port=37153
--webserver_interface=127.7.67.194
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:32:24.539765 28970 log.cc:824] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:24.549659 28970 tablet_bootstrap.cc:492] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:32:24.550169 28970 tablet_bootstrap.cc:492] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0: Bootstrap complete.
I20260806 18:32:24.551714 28970 ts_tablet_manager.cc:1397] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0: Time spent bootstrapping tablet: real 0.035s user 0.022s sys 0.012s
I20260806 18:32:24.555020 28970 raft_consensus.cc:348] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } } peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } }
I20260806 18:32:24.555576 28970 raft_consensus.cc:729] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8fefa6195a948cfa4ed08c816f3e0e0, State: Initialized, Role: FOLLOWER
I20260806 18:32:24.556113 28970 consensus_queue.cc:260] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } } peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } }
I20260806 18:32:24.557186 28970 ts_tablet_manager.cc:1428] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260806 18:32:24.582998 29125 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:32:24.530824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:33999)
0806 18:32:24.532235 (+ 1411us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:24.532301 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:24.533204 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:24.582089 (+ 48885us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1121,"thread_start_us":168,"threads_started":1}
W20260806 18:32:24.583582 29121 heartbeater.cc:644] Failed to heartbeat to 127.7.67.254:34859 (0 consecutive failures): Network error: Failed to ping master at 127.7.67.254:34859: Client connection negotiation failed: client connection to 127.7.67.254:34859: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 18:32:24.584405 29125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:24.584114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:35549)
0806 18:32:24.584252 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:24.584339 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 18:32:24.585100 29125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:24.584885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:52431)
0806 18:32:24.584994 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:24.585044 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 18:32:24.585270 29121 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:32:24.654217 29123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:24.654536 29123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:24.654601 29123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:24.658689 29123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:24.658769 29123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:24.658874 29123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:32:24.664294 29123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:43389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=37153
--tserver_master_addrs=127.7.67.254:34859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:24.665354 29123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:24.666350 29123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:24.676064 29133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:24.676100 29138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:24.676100 29135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:24.676350 29123 server_base.cc:1034] running on GCE node
I20260806 18:32:24.676831 29123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:24.677449 29123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:24.678658 29123 hybrid_clock.cc:648] HybridClock initialized: now 1786041144678619 us; error 49 us; skew 500 ppm
Aug 06 18:32:24 dist-test-slave-5bn2 krb5kdc[27214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041144, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:24.681708 29123 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:32:24.682608 29123 webserver.cc:460] Webserver started at http://127.7.67.194:37153/ using document root <none> and password file <none>
I20260806 18:32:24.683235 29123 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:24.683310 29123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:24.687258 29123 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:32:24.689330 29148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:24.690565 29123 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:32:24.690688 29123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "d4f25ba1e34c4f6095d63a36a6c2babc"
format_stamp: "Formatted at 2026-08-06 18:32:22 on dist-test-slave-5bn2"
server_key: "b273328e413133af883fcc97c178280f"
server_key_iv: "1f97103bcda592e1871cb842aaa161d5"
server_key_version: "encryptionkey@0"
I20260806 18:32:24.691282 29123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:24.701994 29123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:24.705214 29123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:24.705411 29123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:24.706012 29123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:32:24.706875 29123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:32:24.706933 29123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:24.706988 29123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:32:24.707015 29123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:24.715642 29123 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:43389
I20260806 18:32:24.715668 29318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:43389 every 8 connection(s)
I20260806 18:32:24.716640 29123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:32:24.718410 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 29123
I20260806 18:32:24.718815 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:40461
--local_ip_for_outbound_sockets=127.7.67.195
--tserver_master_addrs=127.7.67.254:34859
--webserver_port=46515
--webserver_interface=127.7.67.195
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:32:24.720927 29322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:24.718757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:35481)
0806 18:32:24.719301 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:24.720028 (+ 727us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":145,"threads_started":1}
W20260806 18:32:24.721369 29319 heartbeater.cc:644] Failed to heartbeat to 127.7.67.254:34859 (0 consecutive failures): Network error: Failed to ping master at 127.7.67.254:34859: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
I20260806 18:32:24.721989 29322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:24.721744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:36461)
0806 18:32:24.721865 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:24.721923 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 18:32:24.722579 29322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:24.722369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:53443)
0806 18:32:24.722473 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:24.722522 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 18:32:24.722745 29319 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:32:24.831777 29325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:24.832054 29325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:24.832096 29325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:24.836030 29325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:24.836087 29325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:24.836151 29325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:32:24.841348 29325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:40461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=46515
--tserver_master_addrs=127.7.67.254:34859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:24.842432 29325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:24.843497 29325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:24.853211 29337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:24.853219 29335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:24.853225 29334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:24.853500 29325 server_base.cc:1034] running on GCE node
I20260806 18:32:24.853956 29325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:24.854614 29325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:24.855777 29325 hybrid_clock.cc:648] HybridClock initialized: now 1786041144855753 us; error 34 us; skew 500 ppm
Aug 06 18:32:24 dist-test-slave-5bn2 krb5kdc[27214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041144, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:24.858560 29325 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:32:24.859499 29325 webserver.cc:460] Webserver started at http://127.7.67.195:46515/ using document root <none> and password file <none>
I20260806 18:32:24.860046 29325 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:24.860110 29325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:24.863989 29325 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 18:32:24.866150 29349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:24.867473 29325 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:32:24.867606 29325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "d810a9bcacde44a59767edb24a41bf2d"
format_stamp: "Formatted at 2026-08-06 18:32:22 on dist-test-slave-5bn2"
server_key: "51e2effbbc6f921f03c930f512df7e1f"
server_key_iv: "bb3ed8436dbf018a7afb2ac1f14e0bd9"
server_key_version: "encryptionkey@0"
I20260806 18:32:24.868182 29325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:24.880764 29325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:24.883637 29325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:24.883814 29325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:24.884407 29325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:32:24.885251 29325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:32:24.885304 29325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:24.885361 29325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:32:24.885392 29325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:24.894317 29325 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:40461
I20260806 18:32:24.894342 29530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:40461 every 8 connection(s)
I20260806 18:32:24.895531 29325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:32:24.896260 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 29325
I20260806 18:32:24.899679 29534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:24.897790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:42935)
0806 18:32:24.898238 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:24.898347 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":116,"threads_started":1}
W20260806 18:32:24.900110 29531 heartbeater.cc:644] Failed to heartbeat to 127.7.67.254:34859 (0 consecutive failures): Network error: Failed to ping master at 127.7.67.254:34859: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
I20260806 18:32:24.900725 29534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:24.900511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:39251)
0806 18:32:24.900616 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:24.900664 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 18:32:24.901283 29534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:24.901112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:40865)
0806 18:32:24.901195 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:24.901235 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 18:32:24.901445 29531 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 18:32:25.586424 29536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:25.585915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:58525)
0806 18:32:25.586234 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:25.586300 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":84,"threads_started":1}
I20260806 18:32:25.723814 29537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:25.723365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:58633)
0806 18:32:25.723642 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:25.723700 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":75,"threads_started":1}
I20260806 18:32:25.902405 29540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:25.901993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:34705)
0806 18:32:25.902244 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:25.902302 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":70,"threads_started":1}
I20260806 18:32:26.403141 29544 raft_consensus.cc:482] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:32:26.403366 29544 raft_consensus.cc:504] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } } peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } }
I20260806 18:32:26.404450 29544 leader_election.cc:288] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d810a9bcacde44a59767edb24a41bf2d (127.7.67.195:40461), d4f25ba1e34c4f6095d63a36a6c2babc (127.7.67.194:43389)
Aug 06 18:32:26 dist-test-slave-5bn2 krb5kdc[27214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041144, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.195@KRBTEST.COM
Aug 06 18:32:26 dist-test-slave-5bn2 krb5kdc[27214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041144, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.194@KRBTEST.COM
I20260806 18:32:26.423789 29547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:26.404797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:59917 (local address 127.7.67.195:40461)
0806 18:32:26.405085 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:26.405102 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:26.405149 (+ 47us) server_negotiation.cc:407] Connection header received
0806 18:32:26.405993 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:26.406015 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:26.406492 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:26.406838 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:26.408531 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:26.410095 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:26.411938 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:26.412665 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:26.412849 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:26.419432 (+ 6583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:26.419438 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:26.419452 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:26.419483 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:26.421267 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:26.421747 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:26.421752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:26.421756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:26.421874 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:26.422177 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:26.422180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:26.422182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:26.422736 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:26.422893 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:26.423392 (+ 499us) server_negotiation.cc:299] Negotiation successful
0806 18:32:26.423639 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":83,"threads_started":1}
I20260806 18:32:26.423849 29548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:26.405047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:43389 (local address 127.7.67.193:49523)
0806 18:32:26.405364 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:26.405376 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:26.405473 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:26.407008 (+ 1535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:26.407011 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:26.407535 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:26.408339 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:26.408362 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:26.410247 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:26.410251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:26.411823 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:26.411835 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:26.412791 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:26.412795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:26.413044 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:26.416749 (+ 3705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:26.416777 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:26.419260 (+ 2483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:26.421229 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:26.421236 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:26.421250 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:26.421513 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:26.421844 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:26.421847 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:26.421849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:26.422038 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:26.422757 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:26.422763 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:26.423062 (+ 299us) client_negotiation.cc:769] Sending connection context
0806 18:32:26.423266 (+ 204us) client_negotiation.cc:240] Negotiation successful
0806 18:32:26.423688 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":453,"spinlock_wait_cycles":20992,"thread_start_us":76,"threads_started":1}
I20260806 18:32:26.423863 29545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:26.404694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:40461 (local address 127.7.67.193:59917)
0806 18:32:26.404921 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:26.404936 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:26.405049 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:26.406920 (+ 1871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:26.406925 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:26.407534 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:26.408339 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:26.408362 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:26.410247 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:26.410251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:26.411823 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:26.411835 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:26.412791 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:26.412795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:26.413046 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:26.416749 (+ 3703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:26.416777 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:26.419201 (+ 2424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:26.421382 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:26.421385 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:26.421387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:26.421629 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:26.421965 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:26.421969 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:26.421970 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:26.422067 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:26.422912 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:26.422915 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:26.423062 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 18:32:26.423266 (+ 204us) client_negotiation.cc:240] Negotiation successful
0806 18:32:26.423622 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":288,"spinlock_wait_cycles":29568,"thread_start_us":70,"threads_started":1}
I20260806 18:32:26.424629 29549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:26.405304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:49523 (local address 127.7.67.194:43389)
0806 18:32:26.405584 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:26.405596 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:26.405634 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:32:26.406390 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:26.406398 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:26.406755 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:26.407047 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:26.408530 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:26.410095 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:26.411938 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:26.412665 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:26.412849 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:26.419450 (+ 6601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:26.419456 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:26.419467 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:26.419497 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:26.421112 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:26.421630 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:26.421635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:26.421639 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:26.421753 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:26.422120 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:26.422124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:26.422125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:26.422653 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:26.422785 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:26.424175 (+ 1390us) server_negotiation.cc:299] Negotiation successful
0806 18:32:26.424445 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":72,"threads_started":1}
W20260806 18:32:26.426379 28964 leader_election.cc:341] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d810a9bcacde44a59767edb24a41bf2d (127.7.67.195:40461): Not found: Tablet not found: 5444f1c9d7e14a8bbaa9068f614bcce2
W20260806 18:32:26.426635 28964 leader_election.cc:341] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d4f25ba1e34c4f6095d63a36a6c2babc (127.7.67.194:43389): Not found: Tablet not found: 5444f1c9d7e14a8bbaa9068f614bcce2
I20260806 18:32:26.426716 28964 leader_election.cc:302] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8fefa6195a948cfa4ed08c816f3e0e0; no voters: d4f25ba1e34c4f6095d63a36a6c2babc, d810a9bcacde44a59767edb24a41bf2d
I20260806 18:32:26.426966 29544 raft_consensus.cc:2726] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:32:26.587603 29548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:26.587266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:57131)
0806 18:32:26.587429 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:26.587505 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 18:32:26.725009 29550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:26.724573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:35515)
0806 18:32:26.724836 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:26.724894 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":74,"threads_started":1}
I20260806 18:32:26.903599 29551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:26.903147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:55321)
0806 18:32:26.903411 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:26.903466 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":92,"threads_started":1}
I20260806 18:32:27.588830 29554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:27.588382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:53425)
0806 18:32:27.588663 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:27.588725 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":81,"threads_started":1}
I20260806 18:32:27.726303 29560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:27.725868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:35241)
0806 18:32:27.726141 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:27.726195 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":96,"threads_started":1}
I20260806 18:32:27.904740 29562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:27.904306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:36881)
0806 18:32:27.904583 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:27.904641 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":101,"threads_started":1}
I20260806 18:32:28.204864 29564 raft_consensus.cc:482] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:32:28.205027 29564 raft_consensus.cc:504] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } } peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } }
I20260806 18:32:28.205489 29564 leader_election.cc:288] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d810a9bcacde44a59767edb24a41bf2d (127.7.67.195:40461), d4f25ba1e34c4f6095d63a36a6c2babc (127.7.67.194:43389)
W20260806 18:32:28.206437 28964 leader_election.cc:341] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d810a9bcacde44a59767edb24a41bf2d (127.7.67.195:40461): Not found: Tablet not found: 5444f1c9d7e14a8bbaa9068f614bcce2
W20260806 18:32:28.206600 28964 leader_election.cc:341] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d4f25ba1e34c4f6095d63a36a6c2babc (127.7.67.194:43389): Not found: Tablet not found: 5444f1c9d7e14a8bbaa9068f614bcce2
I20260806 18:32:28.206648 28964 leader_election.cc:302] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8fefa6195a948cfa4ed08c816f3e0e0; no voters: d4f25ba1e34c4f6095d63a36a6c2babc, d810a9bcacde44a59767edb24a41bf2d
I20260806 18:32:28.206796 29564 raft_consensus.cc:2726] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:32:28.590004 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:28.589582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:41029)
0806 18:32:28.589834 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:28.589899 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":81,"threads_started":1}
I20260806 18:32:28.727614 29566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:28.727124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:39871)
0806 18:32:28.727408 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:28.727475 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":82,"threads_started":1}
I20260806 18:32:28.906021 29568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:28.905532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:48623)
0806 18:32:28.905843 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:28.905916 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":80,"threads_started":1}
I20260806 18:32:29.591529 29570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:29.591053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:53057)
0806 18:32:29.591367 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:29.591425 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":82,"threads_started":1}
I20260806 18:32:29.728766 29573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:29.728366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:54461)
0806 18:32:29.728611 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:29.728667 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":79,"threads_started":1}
I20260806 18:32:29.907368 29574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:29.906928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:58873)
0806 18:32:29.907195 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:29.907254 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":89,"threads_started":1}
I20260806 18:32:30.592810 29575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:30.592314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:59895)
0806 18:32:30.592632 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:30.592699 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":88,"threads_started":1}
I20260806 18:32:30.730100 29577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:30.729584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:56825)
0806 18:32:30.729881 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:30.729950 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":98,"threads_started":1}
I20260806 18:32:30.733182 29578 raft_consensus.cc:482] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:32:30.733311 29578 raft_consensus.cc:504] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } } peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } }
I20260806 18:32:30.733692 29578 leader_election.cc:288] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d810a9bcacde44a59767edb24a41bf2d (127.7.67.195:40461), d4f25ba1e34c4f6095d63a36a6c2babc (127.7.67.194:43389)
W20260806 18:32:30.734416 28964 leader_election.cc:341] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d4f25ba1e34c4f6095d63a36a6c2babc (127.7.67.194:43389): Not found: Tablet not found: 5444f1c9d7e14a8bbaa9068f614bcce2
W20260806 18:32:30.734540 28964 leader_election.cc:341] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d810a9bcacde44a59767edb24a41bf2d (127.7.67.195:40461): Not found: Tablet not found: 5444f1c9d7e14a8bbaa9068f614bcce2
I20260806 18:32:30.734592 28964 leader_election.cc:302] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8fefa6195a948cfa4ed08c816f3e0e0; no voters: d4f25ba1e34c4f6095d63a36a6c2babc, d810a9bcacde44a59767edb24a41bf2d
I20260806 18:32:30.734753 29578 raft_consensus.cc:2726] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:32:30.908576 29580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:30.908153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:47599)
0806 18:32:30.908406 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:30.908469 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":96,"threads_started":1}
I20260806 18:32:31.594094 29583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:31.593616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:43095)
0806 18:32:31.593911 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:31.593975 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":81,"threads_started":1}
I20260806 18:32:31.731509 29585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:31.730969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:41863)
0806 18:32:31.731311 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:31.731376 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":76,"threads_started":1}
I20260806 18:32:31.909750 29587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:31.909297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:58439)
0806 18:32:31.909589 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:31.909647 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":75,"threads_started":1}
I20260806 18:32:32.595464 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:32.594941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:59167)
0806 18:32:32.595260 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:32.595323 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":115,"threads_started":1}
I20260806 18:32:32.732736 29590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:32.732300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:45223)
0806 18:32:32.732576 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:32.732636 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":81,"threads_started":1}
I20260806 18:32:32.910909 29591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:32.910465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:59403)
0806 18:32:32.910711 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:32.910770 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":73,"threads_started":1}
I20260806 18:32:33.596675 29592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:33.596226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:36369)
0806 18:32:33.596503 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:33.596565 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":86,"threads_started":1}
I20260806 18:32:33.733927 29593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:33.733514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:39513)
0806 18:32:33.733762 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:33.733823 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":75,"threads_started":1}
I20260806 18:32:33.912180 29594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:33.911694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:38211)
0806 18:32:33.911984 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:33.912054 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":87,"threads_started":1}
I20260806 18:32:34.448972 29595 raft_consensus.cc:482] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:32:34.449131 29595 raft_consensus.cc:504] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d810a9bcacde44a59767edb24a41bf2d" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 40461 } } peers { permanent_uuid: "d4f25ba1e34c4f6095d63a36a6c2babc" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 43389 } } peers { permanent_uuid: "a8fefa6195a948cfa4ed08c816f3e0e0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 38173 } }
I20260806 18:32:34.449544 29595 leader_election.cc:288] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d810a9bcacde44a59767edb24a41bf2d (127.7.67.195:40461), d4f25ba1e34c4f6095d63a36a6c2babc (127.7.67.194:43389)
W20260806 18:32:34.450353 28964 leader_election.cc:341] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d810a9bcacde44a59767edb24a41bf2d (127.7.67.195:40461): Not found: Tablet not found: 5444f1c9d7e14a8bbaa9068f614bcce2
W20260806 18:32:34.450490 28964 leader_election.cc:341] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d4f25ba1e34c4f6095d63a36a6c2babc (127.7.67.194:43389): Not found: Tablet not found: 5444f1c9d7e14a8bbaa9068f614bcce2
I20260806 18:32:34.450547 28964 leader_election.cc:302] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8fefa6195a948cfa4ed08c816f3e0e0; no voters: d4f25ba1e34c4f6095d63a36a6c2babc, d810a9bcacde44a59767edb24a41bf2d
I20260806 18:32:34.450711 29595 raft_consensus.cc:2726] T 5444f1c9d7e14a8bbaa9068f614bcce2 P a8fefa6195a948cfa4ed08c816f3e0e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:32:34.597874 29597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:34.597487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:49209)
0806 18:32:34.597722 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:34.597773 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":80,"threads_started":1}
I20260806 18:32:34.735270 29598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:34.734773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:33509)
0806 18:32:34.735099 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:34.735161 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":135,"threads_started":1}
I20260806 18:32:34.913576 29599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:34.913132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:37785)
0806 18:32:34.913401 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:34.913461 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":78,"threads_started":1}
I20260806 18:32:35.599138 29600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:35.598674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:39291)
0806 18:32:35.598971 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:35.599030 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":85,"threads_started":1}
I20260806 18:32:35.736423 29601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:35.735982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:41605)
0806 18:32:35.736255 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:35.736316 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":75,"threads_started":1}
I20260806 18:32:35.914808 29603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:35.914326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:48013)
0806 18:32:35.914635 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:35.914698 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":99,"threads_started":1}
I20260806 18:32:36.600395 29604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:36.599940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:43479)
0806 18:32:36.600222 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:36.600286 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":82,"threads_started":1}
I20260806 18:32:36.737639 29605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:36.737179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:42225)
0806 18:32:36.737449 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:36.737531 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":78,"threads_started":1}
I20260806 18:32:36.916109 29606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:36.915602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:36723)
0806 18:32:36.915939 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:36.916000 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":78,"threads_started":1}
I20260806 18:32:37.601576 29607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:37.601134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:37057)
0806 18:32:37.601400 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:37.601470 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":77,"threads_started":1}
I20260806 18:32:37.738874 29608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:37.738389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:36469)
0806 18:32:37.738672 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:37.738731 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":78,"threads_started":1}
I20260806 18:32:37.917248 29609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:37.916822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:45561)
0806 18:32:37.917087 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:37.917144 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":98,"threads_started":1}
I20260806 18:32:38.602771 29610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:38.602342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:39801)
0806 18:32:38.602608 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:38.602666 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":86,"threads_started":1}
I20260806 18:32:38.740005 29612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:38.739612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:60093)
0806 18:32:38.739849 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:38.739903 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":78,"threads_started":1}
I20260806 18:32:38.918541 29613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:38.918068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:55511)
0806 18:32:38.918358 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:38.918419 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":91,"threads_started":1}
I20260806 18:32:39.604269 29615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:39.603816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.193:57197)
0806 18:32:39.604099 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:39.604162 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":80,"threads_started":1}
I20260806 18:32:39.741271 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:39.740816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.194:53331)
0806 18:32:39.741116 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:39.741171 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":75,"threads_started":1}
I20260806 18:32:39.903007 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 28882
I20260806 18:32:39.926329 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:39.923151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34859 (local address 127.7.67.195:59319)
0806 18:32:39.926077 (+ 2926us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:39.926170 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2821,"thread_start_us":819,"threads_started":1}
I20260806 18:32:39.934473 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 29123
I20260806 18:32:39.946211 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 29325
2026-08-06T18:32:39Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19502 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:32:40 dist-test-slave-5bn2 krb5kdc[29623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:32:40 dist-test-slave-5bn2 krb5kdc[29623](info): set up 2 sockets
Aug 06 18:32:40 dist-test-slave-5bn2 krb5kdc[29623](info): commencing operation
krb5kdc: starting...
W20260806 18:32:41.076885 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:32:41 dist-test-slave-5bn2 krb5kdc[29623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:32:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:32:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:32:41.199702 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:44587
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:44587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:32:41.312191 29644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:41.312471 29644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:41.312513 29644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:41.316722 29644 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 18:32:41.316778 29644 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 18:32:41.316793 29644 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 18:32:41.316805 29644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:32:41.316821 29644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:41.316854 29644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:41.316874 29644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:32:41.316893 29644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:32:41.322187 29644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:44587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:44587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:41.323316 29644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:41.324352 29644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:41.332772 29655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:41.332801 29650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:41.332779 29651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:41.333060 29644 server_base.cc:1034] running on GCE node
I20260806 18:32:41.333696 29644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:41.334787 29644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:41.335939 29644 hybrid_clock.cc:648] HybridClock initialized: now 1786041161335923 us; error 32 us; skew 500 ppm
Aug 06 18:32:41 dist-test-slave-5bn2 krb5kdc[29623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041161, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:41.338604 29644 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:32:41.339568 29644 webserver.cc:460] Webserver started at http://127.7.67.254:45789/ using document root <none> and password file <none>
I20260806 18:32:41.340126 29644 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:41.340193 29644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:41.340390 29644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:32:41.342453 29644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "a492c393d808401285fe57712a2ecaa3"
format_stamp: "Formatted at 2026-08-06 18:32:41 on dist-test-slave-5bn2"
server_key: "030e90a291d1d7ce1a63064301b285a3"
server_key_iv: "358ae873d13bac407c122f8677e94f23"
server_key_version: "encryptionkey@0"
I20260806 18:32:41.343052 29644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "a492c393d808401285fe57712a2ecaa3"
format_stamp: "Formatted at 2026-08-06 18:32:41 on dist-test-slave-5bn2"
server_key: "030e90a291d1d7ce1a63064301b285a3"
server_key_iv: "358ae873d13bac407c122f8677e94f23"
server_key_version: "encryptionkey@0"
I20260806 18:32:41.347571 29644 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:32:41.349960 29669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:41.351233 29644 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:32:41.351353 29644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "a492c393d808401285fe57712a2ecaa3"
format_stamp: "Formatted at 2026-08-06 18:32:41 on dist-test-slave-5bn2"
server_key: "030e90a291d1d7ce1a63064301b285a3"
server_key_iv: "358ae873d13bac407c122f8677e94f23"
server_key_version: "encryptionkey@0"
I20260806 18:32:41.351459 29644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:41.374828 29644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:41.377869 29644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:41.378075 29644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:41.385419 29644 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:44587
I20260806 18:32:41.385699 29750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:44587 every 8 connection(s)
I20260806 18:32:41.386449 29644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:32:41.389319 29751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:32:41.394928 29751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3: Bootstrap starting.
I20260806 18:32:41.396443 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 29644
I20260806 18:32:41.396579 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:32:41.396957 7439 external_mini_cluster.cc:1428] Setting key 2924ba88bbfbfde430492c692b98af89
I20260806 18:32:41.397941 29751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3: Neither blocks nor log segments found. Creating new log.
I20260806 18:32:41.398809 29751 log.cc:824] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:41.401805 29751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3: No bootstrap required, opened a new log
I20260806 18:32:41.406138 29751 raft_consensus.cc:348] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a492c393d808401285fe57712a2ecaa3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44587 } }
I20260806 18:32:41.406404 29751 raft_consensus.cc:374] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:32:41.406464 29751 raft_consensus.cc:729] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a492c393d808401285fe57712a2ecaa3, State: Initialized, Role: FOLLOWER
I20260806 18:32:41.406961 29751 consensus_queue.cc:260] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a492c393d808401285fe57712a2ecaa3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44587 } }
I20260806 18:32:41.407092 29751 raft_consensus.cc:388] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:32:41.407155 29751 raft_consensus.cc:482] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:32:41.407248 29751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 18:32:41 dist-test-slave-5bn2 krb5kdc[29623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:41.408202 29751 raft_consensus.cc:504] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a492c393d808401285fe57712a2ecaa3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44587 } }
I20260806 18:32:41.408571 29751 leader_election.cc:302] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a492c393d808401285fe57712a2ecaa3; no voters:
I20260806 18:32:41.408905 29751 leader_election.cc:288] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:32:41.409005 29761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:32:41.409260 29761 raft_consensus.cc:686] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 1 LEADER]: Becoming Leader. State: Replica: a492c393d808401285fe57712a2ecaa3, State: Running, Role: LEADER
I20260806 18:32:41.409559 29761 consensus_queue.cc:237] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a492c393d808401285fe57712a2ecaa3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44587 } }
I20260806 18:32:41.410293 29751 sys_catalog.cc:564] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:32:41.410629 29763 sys_catalog.cc:455] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a492c393d808401285fe57712a2ecaa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a492c393d808401285fe57712a2ecaa3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44587 } } }
I20260806 18:32:41.410637 29761 sys_catalog.cc:455] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a492c393d808401285fe57712a2ecaa3. Latest consensus state: current_term: 1 leader_uuid: "a492c393d808401285fe57712a2ecaa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a492c393d808401285fe57712a2ecaa3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44587 } } }
I20260806 18:32:41.410876 29763 sys_catalog.cc:458] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [sys.catalog]: This master's current role is: LEADER
I20260806 18:32:41.410881 29761 sys_catalog.cc:458] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [sys.catalog]: This master's current role is: LEADER
I20260806 18:32:41.412851 29768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:32:41.414043 29757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:41.398235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34146 (local address 127.7.67.254:44587)
0806 18:32:41.400074 (+ 1839us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:41.400091 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:41.400129 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:32:41.400963 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:41.400988 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:41.401392 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:41.401770 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:41.402501 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:41.404046 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:41.404862 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:41.405602 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:41.405767 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:41.408187 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:41.408194 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:41.408210 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:41.408243 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:41.410269 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:41.410820 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:41.410827 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:41.410832 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:41.411193 (+ 361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:41.411504 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:41.411508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:41.411509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:41.412035 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:41.412239 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:41.413013 (+ 774us) server_negotiation.cc:299] Negotiation successful
0806 18:32:41.413313 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1693,"thread_start_us":1517,"threads_started":1}
I20260806 18:32:41.416148 29768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:32:41.421483 29768 catalog_manager.cc:1347] Generated new cluster ID: e2bb8d45ea584d28bd27dce4196efb07
I20260806 18:32:41.421567 29768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:32:41.433821 29768 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:32:41.434798 29768 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:32:41.442065 29768 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3: Generated new TSK 0
I20260806 18:32:41.442723 29768 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
I20260806 18:32:41.477378 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:44587
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:32:41.591310 29791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:41.591583 29791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:41.591625 29791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:41.595588 29791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:41.595649 29791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:41.595706 29791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:32:41.600855 29791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:44587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:41.601890 29791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:41.602906 29791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:41.612403 29803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:41.612413 29799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:41.612538 29791 server_base.cc:1034] running on GCE node
W20260806 18:32:41.612726 29800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:41.613138 29791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:41.613803 29791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:41.614946 29791 hybrid_clock.cc:648] HybridClock initialized: now 1786041161614914 us; error 46 us; skew 500 ppm
Aug 06 18:32:41 dist-test-slave-5bn2 krb5kdc[29623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041161, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:41.617645 29791 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:32:41.618546 29791 webserver.cc:460] Webserver started at http://127.7.67.193:34007/ using document root <none> and password file <none>
I20260806 18:32:41.619140 29791 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:41.619212 29791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:41.619437 29791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:32:41.621484 29791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "251302a9dffd4ce8a7c1330df743f0c1"
format_stamp: "Formatted at 2026-08-06 18:32:41 on dist-test-slave-5bn2"
server_key: "f07c8e03ed5e47f65450d5d38f36f6fa"
server_key_iv: "f5a611975e8fa279bb10bb0c42d9ae2f"
server_key_version: "encryptionkey@0"
I20260806 18:32:41.622045 29791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "251302a9dffd4ce8a7c1330df743f0c1"
format_stamp: "Formatted at 2026-08-06 18:32:41 on dist-test-slave-5bn2"
server_key: "f07c8e03ed5e47f65450d5d38f36f6fa"
server_key_iv: "f5a611975e8fa279bb10bb0c42d9ae2f"
server_key_version: "encryptionkey@0"
I20260806 18:32:41.625857 29791 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 18:32:41.628237 29811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:41.629473 29791 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:32:41.629599 29791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "251302a9dffd4ce8a7c1330df743f0c1"
format_stamp: "Formatted at 2026-08-06 18:32:41 on dist-test-slave-5bn2"
server_key: "f07c8e03ed5e47f65450d5d38f36f6fa"
server_key_iv: "f5a611975e8fa279bb10bb0c42d9ae2f"
server_key_version: "encryptionkey@0"
I20260806 18:32:41.629715 29791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:41.648375 29791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:41.651299 29791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:41.651522 29791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:41.652128 29791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:32:41.652988 29791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:32:41.653040 29791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:41.653095 29791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:32:41.653121 29791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:41.661635 29791 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:36471
I20260806 18:32:41.661666 29977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:36471 every 8 connection(s)
I20260806 18:32:41.662530 29791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:32:41.664793 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 29791
I20260806 18:32:41.664927 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:32:41.665283 7439 external_mini_cluster.cc:1428] Setting key da56a429c7746ddc7e7afff9a51cdcd0
Aug 06 18:32:41 dist-test-slave-5bn2 krb5kdc[29623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041161, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.194@KRBTEST.COM; defaulting to no policy
I20260806 18:32:41.676895 29757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:41.664387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:44997 (local address 127.7.67.254:44587)
0806 18:32:41.664529 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:41.664532 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:41.665741 (+ 1209us) server_negotiation.cc:407] Connection header received
0806 18:32:41.666667 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:41.666671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:41.666722 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:41.666796 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:41.667969 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:41.668690 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:41.670286 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:41.670518 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:41.670588 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:41.673438 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:41.673442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:41.673444 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:41.673469 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:41.674785 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:41.675378 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:41.675380 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:41.675381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:41.675457 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:41.675821 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:41.675823 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:41.675824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:41.675994 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:41.676064 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:41.676662 (+ 598us) server_negotiation.cc:299] Negotiation successful
0806 18:32:41.676770 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:32:41.677716 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:41.664825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:44997)
0806 18:32:41.665580 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:41.665623 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:41.666446 (+ 823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:41.666988 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:41.666995 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:41.667348 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:41.667820 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:41.667833 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:41.668829 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:41.668832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:41.670169 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:41.670181 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:41.670664 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:41.670668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:41.670884 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:41.671471 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:41.671490 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:41.673264 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:41.674931 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:41.674938 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:41.674953 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:41.675285 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:41.675545 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:41.675548 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:41.675550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:41.675734 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:41.676085 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:41.676092 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:41.676420 (+ 328us) client_negotiation.cc:769] Sending connection context
0806 18:32:41.676597 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 18:32:41.676851 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":166,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.194@KRBTEST.COM" created.
I20260806 18:32:41.678996 29978 heartbeater.cc:347] Connected to a master server at 127.7.67.254:44587
I20260806 18:32:41.679235 29978 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:41.679741 29978 heartbeater.cc:511] Master 127.7.67.254:44587 requested a full tablet report, sending...
I20260806 18:32:41.681401 29700 ts_manager.cc:194] Registered new tserver with Master: 251302a9dffd4ce8a7c1330df743f0c1 (127.7.67.193:36471)
I20260806 18:32:41.682665 29700 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:44997
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
I20260806 18:32:41.693588 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:44587
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:32:41.805090 29991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:41.805346 29991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:41.805397 29991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:41.809536 29991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:41.809594 29991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:41.809665 29991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:32:41.814975 29991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:44587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:41.816004 29991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:41.816991 29991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:41.826558 29999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:41.826552 30001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:41.826560 29998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:41.826612 29991 server_base.cc:1034] running on GCE node
I20260806 18:32:41.827255 29991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:41.827874 29991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:41.829031 29991 hybrid_clock.cc:648] HybridClock initialized: now 1786041161829013 us; error 29 us; skew 500 ppm
Aug 06 18:32:41 dist-test-slave-5bn2 krb5kdc[29623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041161, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:41.831749 29991 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:32:41.832811 29991 webserver.cc:460] Webserver started at http://127.7.67.194:42389/ using document root <none> and password file <none>
I20260806 18:32:41.833369 29991 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:41.833439 29991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:41.833622 29991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:32:41.835681 29991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "957e572b0e03411c9808169742a7aed6"
format_stamp: "Formatted at 2026-08-06 18:32:41 on dist-test-slave-5bn2"
server_key: "b18369095d56f9705b9d7ae540d0fcba"
server_key_iv: "64d5615c0c2c17a054b74ae889b4cd59"
server_key_version: "encryptionkey@0"
I20260806 18:32:41.836177 29991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "957e572b0e03411c9808169742a7aed6"
format_stamp: "Formatted at 2026-08-06 18:32:41 on dist-test-slave-5bn2"
server_key: "b18369095d56f9705b9d7ae540d0fcba"
server_key_iv: "64d5615c0c2c17a054b74ae889b4cd59"
server_key_version: "encryptionkey@0"
I20260806 18:32:41.839821 29991 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:32:41.842207 30013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:41.843470 29991 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:32:41.843596 29991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "957e572b0e03411c9808169742a7aed6"
format_stamp: "Formatted at 2026-08-06 18:32:41 on dist-test-slave-5bn2"
server_key: "b18369095d56f9705b9d7ae540d0fcba"
server_key_iv: "64d5615c0c2c17a054b74ae889b4cd59"
server_key_version: "encryptionkey@0"
I20260806 18:32:41.843699 29991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:41.857791 29991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:41.860474 29991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:41.860658 29991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:41.861225 29991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:32:41.862047 29991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:32:41.862097 29991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:41.862152 29991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:32:41.862181 29991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:41.870620 29991 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:46341
I20260806 18:32:41.870640 30194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:46341 every 8 connection(s)
I20260806 18:32:41.871564 29991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:32:41.880604 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 29991
I20260806 18:32:41.880754 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:32:41.881075 7439 external_mini_cluster.cc:1428] Setting key 9ba94323777cd35a71b750cf6afad690
Aug 06 18:32:41 dist-test-slave-5bn2 krb5kdc[29623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041161, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:41.886384 29757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:41.873393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:45455 (local address 127.7.67.254:44587)
0806 18:32:41.873541 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:41.873544 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:41.874377 (+ 833us) server_negotiation.cc:407] Connection header received
0806 18:32:41.875735 (+ 1358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:41.875739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:41.875801 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:41.875875 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:41.877060 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:41.877586 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:41.878800 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:41.879062 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:41.879156 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:41.882039 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:41.882044 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:41.882047 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:41.882079 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:41.883828 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:41.884449 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:41.884453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:41.884455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:41.884630 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:41.885095 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:41.885099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:41.885101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:41.885300 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:41.885399 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:41.886090 (+ 691us) server_negotiation.cc:299] Negotiation successful
0806 18:32:41.886218 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:32:41.887512 30205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:41.873752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:45455)
0806 18:32:41.874228 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:41.874271 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:41.875517 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:41.876069 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:41.876075 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:41.876438 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:41.876914 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:41.876926 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:41.877706 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:41.877709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:41.878667 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:41.878678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:41.879219 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:41.879223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:41.879394 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:41.880036 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:41.880055 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:41.881889 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:41.883966 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:41.883971 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:41.883984 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:41.884331 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:41.884796 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:41.884800 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:41.884802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:41.884995 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:41.885460 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:41.885467 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:41.885801 (+ 334us) client_negotiation.cc:769] Sending connection context
0806 18:32:41.886022 (+ 221us) client_negotiation.cc:240] Negotiation successful
0806 18:32:41.886411 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":134,"threads_started":1}
I20260806 18:32:41.888705 30203 heartbeater.cc:347] Connected to a master server at 127.7.67.254:44587
I20260806 18:32:41.888952 30203 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:41.889441 30203 heartbeater.cc:511] Master 127.7.67.254:44587 requested a full tablet report, sending...
I20260806 18:32:41.890583 29700 ts_manager.cc:194] Registered new tserver with Master: 957e572b0e03411c9808169742a7aed6 (127.7.67.194:46341)
I20260806 18:32:41.891192 29700 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:45455
WARNING: no policy specified for kudu/127.7.67.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
I20260806 18:32:41.910194 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:44587
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:32:42.024065 30212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:32:42.024307 30212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:32:42.024348 30212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:32:42.028301 30212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:32:42.028355 30212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:32:42.028411 30212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:32:42.033560 30212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:44587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:32:42.034668 30212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:32:42.035707 30212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:32:42.045423 30219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:42.045423 30222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:32:42.045415 30220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:32:42.045744 30212 server_base.cc:1034] running on GCE node
I20260806 18:32:42.046141 30212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:32:42.046775 30212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:32:42.047948 30212 hybrid_clock.cc:648] HybridClock initialized: now 1786041162047914 us; error 43 us; skew 500 ppm
Aug 06 18:32:42 dist-test-slave-5bn2 krb5kdc[29623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041162, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:32:42.050655 30212 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:32:42.051568 30212 webserver.cc:460] Webserver started at http://127.7.67.195:38247/ using document root <none> and password file <none>
I20260806 18:32:42.052139 30212 fs_manager.cc:359] Metadata directory not provided
I20260806 18:32:42.052209 30212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:32:42.052416 30212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:32:42.054486 30212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "a60bf7f6826245bf8fcd750e64815f22"
format_stamp: "Formatted at 2026-08-06 18:32:42 on dist-test-slave-5bn2"
server_key: "697495630b1f295187c6dacbfc8a18da"
server_key_iv: "3111ff4e293503468e1c940121d99279"
server_key_version: "encryptionkey@0"
I20260806 18:32:42.055090 30212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "a60bf7f6826245bf8fcd750e64815f22"
format_stamp: "Formatted at 2026-08-06 18:32:42 on dist-test-slave-5bn2"
server_key: "697495630b1f295187c6dacbfc8a18da"
server_key_iv: "3111ff4e293503468e1c940121d99279"
server_key_version: "encryptionkey@0"
I20260806 18:32:42.058820 30212 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:32:42.061478 30231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:42.062691 30212 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:32:42.062808 30212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "a60bf7f6826245bf8fcd750e64815f22"
format_stamp: "Formatted at 2026-08-06 18:32:42 on dist-test-slave-5bn2"
server_key: "697495630b1f295187c6dacbfc8a18da"
server_key_iv: "3111ff4e293503468e1c940121d99279"
server_key_version: "encryptionkey@0"
I20260806 18:32:42.062948 30212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:32:42.082307 30212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:32:42.085357 30212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:32:42.085551 30212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:32:42.086148 30212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:32:42.087006 30212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:32:42.087057 30212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:42.087112 30212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:32:42.087141 30212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:32:42.095391 30212 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:45533
I20260806 18:32:42.095412 30407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:45533 every 8 connection(s)
I20260806 18:32:42.096258 30212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:32:42.097540 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 30212
I20260806 18:32:42.097680 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:32:42.097942 7439 external_mini_cluster.cc:1428] Setting key 435ebf492135037badecf0e1d6a032f0
Aug 06 18:32:42 dist-test-slave-5bn2 krb5kdc[29623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041162, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:32:42.108929 29757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:42.098168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:36249 (local address 127.7.67.254:44587)
0806 18:32:42.098328 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:42.098332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:42.099496 (+ 1164us) server_negotiation.cc:407] Connection header received
0806 18:32:42.100399 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:42.100402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:42.100459 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:42.100529 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:42.101711 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.102245 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.103413 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.103631 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.103717 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:42.106056 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:42.106059 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:42.106061 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:42.106086 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:42.107280 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:42.107736 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:42.107738 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:42.107740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:42.107810 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:42.108054 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:42.108056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:42.108057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:42.108217 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:42.108286 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:42.108702 (+ 416us) server_negotiation.cc:299] Negotiation successful
0806 18:32:42.108803 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 18:32:42.109720 30410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:42.098588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:36249)
0806 18:32:42.099355 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:42.099404 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:42.100186 (+ 782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:42.100705 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:42.100712 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:42.101075 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:32:42.101561 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:42.101575 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.102367 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.102370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:42.103298 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:42.103309 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.103767 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.103772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:42.103934 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:42.104480 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:32:42.104505 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:32:42.105904 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:32:42.107386 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:42.107390 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:42.107402 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:42.107660 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:42.107868 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:32:42.107870 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:32:42.107871 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:32:42.107994 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:32:42.108283 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:32:42.108287 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:32:42.108524 (+ 237us) client_negotiation.cc:769] Sending connection context
0806 18:32:42.108673 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 18:32:42.108917 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":170,"threads_started":1}
I20260806 18:32:42.110895 30408 heartbeater.cc:347] Connected to a master server at 127.7.67.254:44587
I20260806 18:32:42.111145 30408 heartbeater.cc:464] Registering TS with master...
I20260806 18:32:42.111760 30408 heartbeater.cc:511] Master 127.7.67.254:44587 requested a full tablet report, sending...
I20260806 18:32:42.112780 29700 ts_manager.cc:194] Registered new tserver with Master: a60bf7f6826245bf8fcd750e64815f22 (127.7.67.195:45533)
I20260806 18:32:42.113324 29700 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:36249
I20260806 18:32:42.123581 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:32:42.132069 29757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:42.125453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34150 (local address 127.7.67.254:44587)
0806 18:32:42.125572 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:42.125576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:42.125746 (+ 170us) server_negotiation.cc:407] Connection header received
0806 18:32:42.125842 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:42.125844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:42.125882 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:42.125948 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:32:42.126566 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.127094 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.127853 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.128100 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.128174 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:42.129370 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:32:42.129373 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:32:42.129375 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:32:42.129399 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:32:42.130639 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:42.131070 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:42.131074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:42.131075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:42.131165 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:32:42.131366 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:32:42.131368 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:32:42.131369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:32:42.131587 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:32:42.131692 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:42.131845 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 18:32:42.131943 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:32:42.134742 29700 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:32:42.137811 29700 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:32:42.152719 30435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:42.147558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40622 (local address 127.7.67.193:36471)
0806 18:32:42.147847 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:42.147862 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:42.148022 (+ 160us) server_negotiation.cc:407] Connection header received
0806 18:32:42.148136 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:42.148141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:42.148326 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:42.148498 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:42.148663 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.149727 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.151182 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.152018 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.152133 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:42.152293 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:42.152462 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 18:32:42.152564 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":97,"threads_started":1}
I20260806 18:32:42.152711 30431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:42.147115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:36471 (local address 127.0.0.1:40622)
0806 18:32:42.147921 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:42.147934 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:42.148037 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:42.148469 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:42.148472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:42.148487 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:42.148581 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:42.148586 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.149852 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.149856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:42.151065 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:42.151076 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.152151 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.152154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:42.152268 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:42.152307 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 18:32:42.152419 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 18:32:42.152546 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":68,"thread_start_us":99,"threads_started":1}
I20260806 18:32:42.153023 30433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:42.147019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54574 (local address 127.7.67.194:46341)
0806 18:32:42.148251 (+ 1232us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:42.148259 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:42.148277 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:32:42.148345 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:42.148350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:42.148537 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:42.148690 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:42.149033 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.149789 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.151478 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.152037 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.152132 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:42.152292 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:42.152775 (+ 483us) server_negotiation.cc:299] Negotiation successful
0806 18:32:42.152882 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1147,"thread_start_us":150,"threads_started":1}
I20260806 18:32:42.153294 30430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:42.146196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:45533 (local address 127.0.0.1:39516)
0806 18:32:42.147053 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:42.147107 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:42.147253 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:42.148088 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:42.148094 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:42.148120 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:42.148247 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:42.148253 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.149777 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.149780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:42.151217 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:42.151228 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.152368 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.152372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:42.152534 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:42.152559 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:32:42.152629 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 18:32:42.152695 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"mutex_wait_us":29,"thread_start_us":73,"threads_started":1}
I20260806 18:32:42.153326 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:42.146330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39516 (local address 127.7.67.195:45533)
0806 18:32:42.146778 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:42.146791 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:42.147392 (+ 601us) server_negotiation.cc:407] Connection header received
0806 18:32:42.147476 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:42.147481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:42.147708 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:42.147889 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:42.148407 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.149628 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.151337 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.152110 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.152197 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:42.152306 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:42.152799 (+ 493us) server_negotiation.cc:299] Negotiation successful
0806 18:32:42.153181 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":159,"threads_started":1}
I20260806 18:32:42.153541 30434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:42.146298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:46341 (local address 127.0.0.1:54574)
0806 18:32:42.147387 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:42.147400 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:42.147490 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:42.148839 (+ 1349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:42.148841 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:42.148856 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:42.148917 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:42.148921 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.149960 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.149963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:42.151371 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:42.151379 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.152353 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.152356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:42.152501 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:42.152532 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 18:32:42.152595 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 18:32:42.152652 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1029,"mutex_wait_us":189,"thread_start_us":114,"threads_started":1}
I20260806 18:32:42.156273 30304 tablet_service.cc:1467] Processing CreateTablet for tablet 7ef2f72b778446179e4acc89fdea8d0d (DEFAULT_TABLE table=test-table [id=f99baa55d3b5464ba58e9b8a79c7b1d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:32:42.156349 29884 tablet_service.cc:1467] Processing CreateTablet for tablet 7ef2f72b778446179e4acc89fdea8d0d (DEFAULT_TABLE table=test-table [id=f99baa55d3b5464ba58e9b8a79c7b1d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:32:42.156348 30095 tablet_service.cc:1467] Processing CreateTablet for tablet 7ef2f72b778446179e4acc89fdea8d0d (DEFAULT_TABLE table=test-table [id=f99baa55d3b5464ba58e9b8a79c7b1d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:32:42.157408 30304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ef2f72b778446179e4acc89fdea8d0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:32:42.157418 29884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ef2f72b778446179e4acc89fdea8d0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:32:42.157419 30095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ef2f72b778446179e4acc89fdea8d0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:32:42.163503 30436 tablet_bootstrap.cc:492] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22: Bootstrap starting.
I20260806 18:32:42.165581 30438 tablet_bootstrap.cc:492] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1: Bootstrap starting.
I20260806 18:32:42.166386 30436 tablet_bootstrap.cc:654] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22: Neither blocks nor log segments found. Creating new log.
I20260806 18:32:42.167217 30436 log.cc:824] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:42.167229 30437 tablet_bootstrap.cc:492] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6: Bootstrap starting.
I20260806 18:32:42.167624 30438 tablet_bootstrap.cc:654] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1: Neither blocks nor log segments found. Creating new log.
I20260806 18:32:42.168866 30438 log.cc:824] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:42.169077 30436 tablet_bootstrap.cc:492] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22: No bootstrap required, opened a new log
I20260806 18:32:42.169277 30436 ts_tablet_manager.cc:1397] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:32:42.169937 30437 tablet_bootstrap.cc:654] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6: Neither blocks nor log segments found. Creating new log.
I20260806 18:32:42.170769 30437 log.cc:824] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:32:42.171370 30438 tablet_bootstrap.cc:492] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1: No bootstrap required, opened a new log
I20260806 18:32:42.171557 30438 ts_tablet_manager.cc:1397] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:32:42.172806 30437 tablet_bootstrap.cc:492] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6: No bootstrap required, opened a new log
I20260806 18:32:42.172752 30436 raft_consensus.cc:348] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:32:42.172986 30437 ts_tablet_manager.cc:1397] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260806 18:32:42.173040 30436 raft_consensus.cc:374] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:32:42.173096 30436 raft_consensus.cc:729] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a60bf7f6826245bf8fcd750e64815f22, State: Initialized, Role: FOLLOWER
I20260806 18:32:42.173521 30436 consensus_queue.cc:260] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:32:42.174176 30408 heartbeater.cc:503] Master 127.7.67.254:44587 was elected leader, sending a full tablet report...
I20260806 18:32:42.174546 30436 ts_tablet_manager.cc:1428] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:32:42.175328 30438 raft_consensus.cc:348] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:32:42.175599 30438 raft_consensus.cc:374] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:32:42.175660 30438 raft_consensus.cc:729] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 251302a9dffd4ce8a7c1330df743f0c1, State: Initialized, Role: FOLLOWER
I20260806 18:32:42.176120 30438 consensus_queue.cc:260] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:32:42.176828 29978 heartbeater.cc:503] Master 127.7.67.254:44587 was elected leader, sending a full tablet report...
I20260806 18:32:42.176730 30437 raft_consensus.cc:348] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:32:42.176975 30437 raft_consensus.cc:374] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:32:42.177034 30437 raft_consensus.cc:729] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 957e572b0e03411c9808169742a7aed6, State: Initialized, Role: FOLLOWER
I20260806 18:32:42.177142 30438 ts_tablet_manager.cc:1428] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:32:42.177460 30437 consensus_queue.cc:260] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:32:42.178157 30203 heartbeater.cc:503] Master 127.7.67.254:44587 was elected leader, sending a full tablet report...
I20260806 18:32:42.178403 30437 ts_tablet_manager.cc:1428] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20260806 18:32:42.191732 30447 raft_consensus.cc:482] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:32:42.191924 30447 raft_consensus.cc:504] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:32:42.193011 30447 leader_election.cc:288] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a60bf7f6826245bf8fcd750e64815f22 (127.7.67.195:45533), 251302a9dffd4ce8a7c1330df743f0c1 (127.7.67.193:36471)
I20260806 18:32:42.196574 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:42.193340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:34361 (local address 127.7.67.195:45533)
0806 18:32:42.193487 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:42.193491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:42.193518 (+ 27us) server_negotiation.cc:407] Connection header received
0806 18:32:42.193596 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:42.193600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:42.193647 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:42.193708 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:42.193976 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.194481 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.195648 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.196050 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.196159 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:42.196213 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:42.196387 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 18:32:42.196456 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:32:42.196578 30205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:42.193231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:45533 (local address 127.7.67.194:34361)
0806 18:32:42.193403 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:42.193419 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:42.193504 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:42.193733 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:42.193735 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:42.193764 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:42.193870 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:42.193876 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.194604 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.194607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:42.195521 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:42.195531 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.196166 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.196169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:42.196273 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:42.196292 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:32:42.196358 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 18:32:42.196431 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":38,"spinlock_wait_cycles":1280}
I20260806 18:32:42.196681 30435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:42.193343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:33731 (local address 127.7.67.193:36471)
0806 18:32:42.193505 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:42.193510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:42.193736 (+ 226us) server_negotiation.cc:407] Connection header received
0806 18:32:42.193837 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:42.193841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:42.193883 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:42.193942 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:32:42.194209 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.194693 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.195778 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.196163 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.196267 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:42.196318 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:42.196487 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 18:32:42.196567 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 18:32:42.196722 30449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:42.193252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:36471 (local address 127.7.67.194:33731)
0806 18:32:42.193644 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:42.193655 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:32:42.193754 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:32:42.193962 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:32:42.193964 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:32:42.193979 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:32:42.194098 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:42.194103 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.194794 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.194797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:42.195656 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:32:42.195666 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.196266 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.196270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:32:42.196373 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:42.196398 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:32:42.196460 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 18:32:42.196531 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":10,"thread_start_us":112,"threads_started":1}
I20260806 18:32:42.197269 30343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef2f72b778446179e4acc89fdea8d0d" candidate_uuid: "957e572b0e03411c9808169742a7aed6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a60bf7f6826245bf8fcd750e64815f22" is_pre_election: true
I20260806 18:32:42.197269 29911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef2f72b778446179e4acc89fdea8d0d" candidate_uuid: "957e572b0e03411c9808169742a7aed6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "251302a9dffd4ce8a7c1330df743f0c1" is_pre_election: true
I20260806 18:32:42.197599 29911 raft_consensus.cc:2454] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 957e572b0e03411c9808169742a7aed6 in term 0.
I20260806 18:32:42.197599 30343 raft_consensus.cc:2454] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 957e572b0e03411c9808169742a7aed6 in term 0.
I20260806 18:32:42.198091 30017 leader_election.cc:302] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 957e572b0e03411c9808169742a7aed6, a60bf7f6826245bf8fcd750e64815f22; no voters:
I20260806 18:32:42.198351 30447 raft_consensus.cc:2781] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:32:42.198439 30447 raft_consensus.cc:482] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:32:42.198490 30447 raft_consensus.cc:3037] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:32:42.199599 30447 raft_consensus.cc:504] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:32:42.199928 30447 leader_election.cc:288] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [CANDIDATE]: Term 1 election: Requested vote from peers a60bf7f6826245bf8fcd750e64815f22 (127.7.67.195:45533), 251302a9dffd4ce8a7c1330df743f0c1 (127.7.67.193:36471)
I20260806 18:32:42.200285 30343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef2f72b778446179e4acc89fdea8d0d" candidate_uuid: "957e572b0e03411c9808169742a7aed6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a60bf7f6826245bf8fcd750e64815f22"
I20260806 18:32:42.200304 29911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef2f72b778446179e4acc89fdea8d0d" candidate_uuid: "957e572b0e03411c9808169742a7aed6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "251302a9dffd4ce8a7c1330df743f0c1"
I20260806 18:32:42.200408 29911 raft_consensus.cc:3037] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:32:42.200408 30343 raft_consensus.cc:3037] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:32:42.201520 29911 raft_consensus.cc:2454] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 957e572b0e03411c9808169742a7aed6 in term 1.
I20260806 18:32:42.201576 30343 raft_consensus.cc:2454] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 957e572b0e03411c9808169742a7aed6 in term 1.
I20260806 18:32:42.201889 30015 leader_election.cc:302] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 251302a9dffd4ce8a7c1330df743f0c1, 957e572b0e03411c9808169742a7aed6, a60bf7f6826245bf8fcd750e64815f22; no voters:
I20260806 18:32:42.202105 30447 raft_consensus.cc:2781] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:32:42.202340 30447 raft_consensus.cc:686] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [term 1 LEADER]: Becoming Leader. State: Replica: 957e572b0e03411c9808169742a7aed6, State: Running, Role: LEADER
I20260806 18:32:42.202670 30447 consensus_queue.cc:237] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:32:42.205931 29700 catalog_manager.cc:5495] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 957e572b0e03411c9808169742a7aed6 (127.7.67.194). New cstate: current_term: 1 leader_uuid: "957e572b0e03411c9808169742a7aed6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } health_report { overall_health: UNKNOWN } } }
I20260806 18:32:42.222352 30433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:42.219255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54588 (local address 127.7.67.194:46341)
0806 18:32:42.219377 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:32:42.219380 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:32:42.219394 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:32:42.219533 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:32:42.219535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:32:42.219596 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:32:42.219686 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:32:42.219866 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.220385 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.221153 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:32:42.221351 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:32:42.221415 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:32:42.221642 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:32:42.222074 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:32:42.222162 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:32:42.222186 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 18:32:42.222237 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:32:42.228246 29911 raft_consensus.cc:1261] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Refusing update from remote peer 957e572b0e03411c9808169742a7aed6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:32:42.228246 30343 raft_consensus.cc:1261] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22 [term 1 FOLLOWER]: Refusing update from remote peer 957e572b0e03411c9808169742a7aed6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:32:42.228961 30450 consensus_queue.cc:1035] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:32:42.229146 30447 consensus_queue.cc:1035] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:32:42.240211 30457 mvcc.cc:205] Tried to move back new op lower bound from 7315624600479830016 to 7315624600384819200. Current Snapshot: MvccSnapshot[applied={T|T < 7315624600479830016}]
I20260806 18:32:42.240410 30456 mvcc.cc:205] Tried to move back new op lower bound from 7315624600479830016 to 7315624600384819200. Current Snapshot: MvccSnapshot[applied={T|T < 7315624600479830016 or (T in {7315624600479830016})}]
I20260806 18:32:42.244544 29699 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:32:42.244741 29699 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:32:42.247345 29699 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3: Sending DeleteTablet for 3 replicas of tablet 7ef2f72b778446179e4acc89fdea8d0d
I20260806 18:32:42.248178 30304 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ef2f72b778446179e4acc89fdea8d0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:32:42 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:39516
I20260806 18:32:42.248454 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 29791
I20260806 18:32:42.248445 29884 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ef2f72b778446179e4acc89fdea8d0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:32:42 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:40622
I20260806 18:32:42.248710 30095 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ef2f72b778446179e4acc89fdea8d0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:32:42 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:54574
I20260806 18:32:42.257206 30472 tablet_replica.cc:330] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6: stopping tablet replica
I20260806 18:32:42.257632 30472 raft_consensus.cc:2229] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:32:42.257869 30470 tablet_replica.cc:330] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22: stopping tablet replica
I20260806 18:32:42.258087 30472 raft_consensus.cc:2258] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:32:42.258311 30470 raft_consensus.cc:2229] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:32:42.258723 30470 raft_consensus.cc:2258] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:32:42.260488 30472 ts_tablet_manager.cc:1905] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:32:42.261431 30470 ts_tablet_manager.cc:1905] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:32:42.263605 30472 ts_tablet_manager.cc:1918] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:32:42.263792 30472 log.cc:1196] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/7ef2f72b778446179e4acc89fdea8d0d
I20260806 18:32:42.264109 30470 ts_tablet_manager.cc:1918] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:32:42.264210 30470 log.cc:1196] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/7ef2f72b778446179e4acc89fdea8d0d
I20260806 18:32:42.264334 30472 ts_tablet_manager.cc:1939] T 7ef2f72b778446179e4acc89fdea8d0d P 957e572b0e03411c9808169742a7aed6: Deleting consensus metadata
I20260806 18:32:42.264494 30470 ts_tablet_manager.cc:1939] T 7ef2f72b778446179e4acc89fdea8d0d P a60bf7f6826245bf8fcd750e64815f22: Deleting consensus metadata
I20260806 18:32:42.265777 29670 catalog_manager.cc:4841] TS 957e572b0e03411c9808169742a7aed6 (127.7.67.194:46341): tablet 7ef2f72b778446179e4acc89fdea8d0d (table test-table [id=f99baa55d3b5464ba58e9b8a79c7b1d2]) successfully deleted
I20260806 18:32:42.265779 29676 catalog_manager.cc:4841] TS a60bf7f6826245bf8fcd750e64815f22 (127.7.67.195:45533): tablet 7ef2f72b778446179e4acc89fdea8d0d (table test-table [id=f99baa55d3b5464ba58e9b8a79c7b1d2]) successfully deleted
W20260806 18:32:42.267863 29673 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.193:36471 (error 108)
I20260806 18:32:42.268772 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 29991
I20260806 18:32:42.268980 30434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:32:42.268660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:36471 (local address 127.0.0.1:40630)
0806 18:32:42.268810 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 18:32:42.268888 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:36471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 18:32:42.272351 29673 catalog_manager.cc:4584] TS 251302a9dffd4ce8a7c1330df743f0c1 (127.7.67.193:36471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7ef2f72b778446179e4acc89fdea8d0d: Network error: Client connection negotiation failed: client connection to 127.7.67.193:36471: connect: Connection refused (error 111)
I20260806 18:32:42.281991 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 30212
I20260806 18:32:42.292893 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 29644
I20260806 18:33:02.303668 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:44587
--webserver_interface=127.7.67.254
--webserver_port=45789
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:44587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:33:02.424746 30475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:02.425060 30475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:02.425103 30475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:02.429191 30475 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 18:33:02.429267 30475 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 18:33:02.429286 30475 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 18:33:02.429301 30475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:33:02.429322 30475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:02.429342 30475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:02.429368 30475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:33:02.429392 30475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:33:02.435003 30475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:44587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:44587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=45789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:02.436137 30475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:02.437176 30475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:02.451838 30486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:02.451906 30484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:02.451891 30475 server_base.cc:1034] running on GCE node
W20260806 18:33:02.451839 30488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:02.452873 30475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:02.454104 30475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:02.455274 30475 hybrid_clock.cc:648] HybridClock initialized: now 1786041182455254 us; error 33 us; skew 500 ppm
Aug 06 18:33:02 dist-test-slave-5bn2 krb5kdc[29623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041182, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:02.458190 30475 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:33:02.459199 30475 webserver.cc:460] Webserver started at http://127.7.67.254:45789/ using document root <none> and password file <none>
I20260806 18:33:02.459782 30475 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:02.459858 30475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:02.463967 30475 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:33:02.466295 30496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:02.467694 30475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:33:02.467823 30475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "a492c393d808401285fe57712a2ecaa3"
format_stamp: "Formatted at 2026-08-06 18:32:41 on dist-test-slave-5bn2"
server_key: "030e90a291d1d7ce1a63064301b285a3"
server_key_iv: "358ae873d13bac407c122f8677e94f23"
server_key_version: "encryptionkey@0"
I20260806 18:33:02.468472 30475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:02.485193 30475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:02.488351 30475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:02.488538 30475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:02.495908 30475 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:44587
I20260806 18:33:02.496199 30584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:44587 every 8 connection(s)
I20260806 18:33:02.497109 30475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:33:02.501874 30585 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:33:02.502404 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 30475
I20260806 18:33:02.502966 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:36471
--local_ip_for_outbound_sockets=127.7.67.193
--tserver_master_addrs=127.7.67.254:44587
--webserver_port=34007
--webserver_interface=127.7.67.193
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:33:02.504240 30585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3: Bootstrap starting.
I20260806 18:33:02.522594 30585 log.cc:824] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:02.532092 30585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:33:02.532470 30585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3: Bootstrap complete.
I20260806 18:33:02.536664 30585 raft_consensus.cc:348] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a492c393d808401285fe57712a2ecaa3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44587 } }
I20260806 18:33:02.537111 30585 raft_consensus.cc:729] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a492c393d808401285fe57712a2ecaa3, State: Initialized, Role: FOLLOWER
I20260806 18:33:02.537601 30585 consensus_queue.cc:260] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a492c393d808401285fe57712a2ecaa3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44587 } }
I20260806 18:33:02.537730 30585 raft_consensus.cc:388] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:33:02.537806 30585 raft_consensus.cc:482] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:33:02.537902 30585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:33:02.540009 30585 raft_consensus.cc:504] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a492c393d808401285fe57712a2ecaa3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44587 } }
I20260806 18:33:02.540351 30585 leader_election.cc:302] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a492c393d808401285fe57712a2ecaa3; no voters:
I20260806 18:33:02.540670 30585 leader_election.cc:288] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:33:02.540776 30590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:33:02.540973 30590 raft_consensus.cc:686] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 2 LEADER]: Becoming Leader. State: Replica: a492c393d808401285fe57712a2ecaa3, State: Running, Role: LEADER
I20260806 18:33:02.541361 30590 consensus_queue.cc:237] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a492c393d808401285fe57712a2ecaa3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44587 } }
I20260806 18:33:02.542078 30585 sys_catalog.cc:564] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:33:02.542831 30593 sys_catalog.cc:455] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a492c393d808401285fe57712a2ecaa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a492c393d808401285fe57712a2ecaa3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44587 } } }
I20260806 18:33:02.543278 30593 sys_catalog.cc:458] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:02.543183 30594 sys_catalog.cc:455] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a492c393d808401285fe57712a2ecaa3. Latest consensus state: current_term: 2 leader_uuid: "a492c393d808401285fe57712a2ecaa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a492c393d808401285fe57712a2ecaa3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44587 } } }
I20260806 18:33:02.543649 30594 sys_catalog.cc:458] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:02.544001 30606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:33:02.549183 30606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:33:02.549902 30606 catalog_manager.cc:1259] Loaded cluster ID: e2bb8d45ea584d28bd27dce4196efb07
I20260806 18:33:02.549973 30606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:33:02.552520 30606 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:33:02.553227 30606 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3: Loaded TSK: 0
I20260806 18:33:02.572192 30606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3: Generated new TSK 1
I20260806 18:33:02.572669 30606 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:33:02.627682 30587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:02.627945 30587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:02.627988 30587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:02.632066 30587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:02.632130 30587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:02.632189 30587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:33:02.637408 30587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:36471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=34007
--tserver_master_addrs=127.7.67.254:44587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:02.638468 30587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:02.639518 30587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:02.649243 30622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:02.649401 30630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:02.649216 30625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:02.649576 30587 server_base.cc:1034] running on GCE node
I20260806 18:33:02.650099 30587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:02.650753 30587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:02.651934 30587 hybrid_clock.cc:648] HybridClock initialized: now 1786041182651909 us; error 41 us; skew 500 ppm
Aug 06 18:33:02 dist-test-slave-5bn2 krb5kdc[29623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041182, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:02.654878 30587 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:33:02.655779 30587 webserver.cc:460] Webserver started at http://127.7.67.193:34007/ using document root <none> and password file <none>
I20260806 18:33:02.656332 30587 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:02.656428 30587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:02.660586 30587 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:33:02.662926 30643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:02.664225 30587 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:33:02.664354 30587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "251302a9dffd4ce8a7c1330df743f0c1"
format_stamp: "Formatted at 2026-08-06 18:32:41 on dist-test-slave-5bn2"
server_key: "f07c8e03ed5e47f65450d5d38f36f6fa"
server_key_iv: "f5a611975e8fa279bb10bb0c42d9ae2f"
server_key_version: "encryptionkey@0"
I20260806 18:33:02.664913 30587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:02.681478 30587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:02.684705 30587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:02.684895 30587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:02.685505 30587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:02.686599 30653 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:33:02.690949 30587 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:33:02.691030 30587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260806 18:33:02.691107 30587 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:33:02.692938 30587 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:33:02.693001 30587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 18:33:02.693285 30653 tablet_bootstrap.cc:492] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1: Bootstrap starting.
I20260806 18:33:02.703508 30587 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:36471
I20260806 18:33:02.703574 30826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:36471 every 8 connection(s)
I20260806 18:33:02.704540 30587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:33:02.711174 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 30587
I20260806 18:33:02.711723 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:46341
--local_ip_for_outbound_sockets=127.7.67.194
--tserver_master_addrs=127.7.67.254:44587
--webserver_port=42389
--webserver_interface=127.7.67.194
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:33:02.713091 30653 log.cc:824] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:33:02 dist-test-slave-5bn2 krb5kdc[29623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041182, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:33:02.721298 30653 tablet_bootstrap.cc:492] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:33:02.721746 30653 tablet_bootstrap.cc:492] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1: Bootstrap complete.
I20260806 18:33:02.722947 30653 ts_tablet_manager.cc:1397] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1: Time spent bootstrapping tablet: real 0.030s user 0.026s sys 0.004s
I20260806 18:33:02.724118 30829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:02.706551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:58971)
0806 18:33:02.707168 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:02.707228 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:02.708372 (+ 1144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:02.709601 (+ 1229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:02.709611 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:02.710154 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:02.710958 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:02.710985 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:02.712820 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:02.712824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:02.714172 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:02.714183 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:02.715008 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:02.715012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:02.715195 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:02.715884 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:33:02.715905 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:02.717744 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:02.720368 (+ 2624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:02.720375 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:02.720386 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:02.720737 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:02.721177 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:02.721181 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:02.721183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:02.721349 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:02.722108 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:02.722115 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:02.722432 (+ 317us) client_negotiation.cc:769] Sending connection context
0806 18:33:02.722657 (+ 225us) client_negotiation.cc:240] Negotiation successful
0806 18:33:02.723129 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":183,"threads_started":1}
I20260806 18:33:02.724136 30832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:02.706705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:58971 (local address 127.7.67.254:44587)
0806 18:33:02.707237 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:02.707259 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:02.707343 (+ 84us) server_negotiation.cc:407] Connection header received
0806 18:33:02.708812 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:02.708840 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:02.709145 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:02.709446 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:02.711194 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:02.712641 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:02.714303 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:02.714804 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:02.715036 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:02.717978 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:02.717986 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:02.718015 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:02.718053 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:02.720228 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:02.720864 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:02.720871 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:02.720877 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:02.721060 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:02.721478 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:02.721481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:02.721483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:02.721972 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:02.722109 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:02.722782 (+ 673us) server_negotiation.cc:299] Negotiation successful
0806 18:33:02.723124 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":151,"threads_started":1}
I20260806 18:33:02.725723 30653 raft_consensus.cc:348] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:33:02.726231 30653 raft_consensus.cc:729] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 251302a9dffd4ce8a7c1330df743f0c1, State: Initialized, Role: FOLLOWER
I20260806 18:33:02.726714 30653 consensus_queue.cc:260] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:33:02.727190 30827 heartbeater.cc:347] Connected to a master server at 127.7.67.254:44587
I20260806 18:33:02.727411 30827 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:02.727767 30653 ts_tablet_manager.cc:1428] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:33:02.727962 30827 heartbeater.cc:511] Master 127.7.67.254:44587 requested a full tablet report, sending...
I20260806 18:33:02.730314 30534 ts_manager.cc:194] Registered new tserver with Master: 251302a9dffd4ce8a7c1330df743f0c1 (127.7.67.193:36471)
I20260806 18:33:02.733632 30534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:58971
I20260806 18:33:02.736132 30827 heartbeater.cc:503] Master 127.7.67.254:44587 was elected leader, sending a full tablet report...
Aug 06 18:33:02 dist-test-slave-5bn2 krb5kdc[29623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041182, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for kudu/127.7.67.193@KRBTEST.COM
I20260806 18:33:02.743847 30841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:02.733124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48498 (local address 127.7.67.193:36471)
0806 18:33:02.733495 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:02.733517 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:02.733937 (+ 420us) server_negotiation.cc:407] Connection header received
0806 18:33:02.734130 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:02.734135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:02.734344 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:02.734497 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:02.735254 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:02.736279 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:02.737772 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:02.738100 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:02.738181 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:02.740525 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:02.740534 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:02.740543 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:02.740565 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:02.741982 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:02.742434 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:02.742439 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:02.742440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:02.742519 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:02.742837 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:02.742840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:02.742841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:02.743113 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:02.743227 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:02.743582 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 18:33:02.743720 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":109,"threads_started":1}
I20260806 18:33:02.743878 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:02.733022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:36471 (local address 127.0.0.1:48498)
0806 18:33:02.733785 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:02.733833 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:02.733976 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:02.734515 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:02.734521 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:02.734978 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:02.735122 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:02.735131 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:02.736642 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:02.736646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:02.737664 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:02.737676 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:02.738206 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:02.738210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:02.738318 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:02.738838 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:33:02.738891 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:02.740373 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:02.742088 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:02.742097 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:02.742103 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:02.742320 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:02.742629 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:02.742632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:02.742634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:02.742740 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:02.743227 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:02.743233 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:02.743365 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 18:33:02.743512 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 18:33:02.743709 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":111,"threads_started":1}
I20260806 18:33:02.745096 30720 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ef2f72b778446179e4acc89fdea8d0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:32:42 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:48498
I20260806 18:33:02.746004 30844 tablet_replica.cc:330] stopping tablet replica
I20260806 18:33:02.746332 30844 raft_consensus.cc:2229] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:33:02.746593 30844 raft_consensus.cc:2258] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:33:02.747809 30844 ts_tablet_manager.cc:1905] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:33:02.752486 30844 ts_tablet_manager.cc:1918] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:33:02.752599 30844 log.cc:1196] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/wal/wals/7ef2f72b778446179e4acc89fdea8d0d
I20260806 18:33:02.752869 30844 ts_tablet_manager.cc:1939] T 7ef2f72b778446179e4acc89fdea8d0d P 251302a9dffd4ce8a7c1330df743f0c1: Deleting consensus metadata
I20260806 18:33:02.754953 30507 catalog_manager.cc:4841] TS 251302a9dffd4ce8a7c1330df743f0c1 (127.7.67.193:36471): tablet 7ef2f72b778446179e4acc89fdea8d0d (table test-table [id=f99baa55d3b5464ba58e9b8a79c7b1d2]) successfully deleted
W20260806 18:33:02.833253 30835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:02.833523 30835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:02.833566 30835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:02.837500 30835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:02.837559 30835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:02.837613 30835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:33:02.843004 30835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:46341
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=42389
--tserver_master_addrs=127.7.67.254:44587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:02.844103 30835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:02.845160 30835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:02.855050 30851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:02.854995 30854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:02.855223 30835 server_base.cc:1034] running on GCE node
W20260806 18:33:02.855036 30852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:02.855633 30835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:02.856295 30835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:02.857446 30835 hybrid_clock.cc:648] HybridClock initialized: now 1786041182857415 us; error 45 us; skew 500 ppm
Aug 06 18:33:02 dist-test-slave-5bn2 krb5kdc[29623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041182, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:02.860227 30835 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:33:02.861123 30835 webserver.cc:460] Webserver started at http://127.7.67.194:42389/ using document root <none> and password file <none>
I20260806 18:33:02.861833 30835 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:02.861940 30835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:02.865813 30835 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:33:02.867857 30863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:02.869117 30835 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:33:02.869230 30835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "957e572b0e03411c9808169742a7aed6"
format_stamp: "Formatted at 2026-08-06 18:32:41 on dist-test-slave-5bn2"
server_key: "b18369095d56f9705b9d7ae540d0fcba"
server_key_iv: "64d5615c0c2c17a054b74ae889b4cd59"
server_key_version: "encryptionkey@0"
I20260806 18:33:02.869762 30835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:02.888485 30835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:02.891386 30835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:02.891587 30835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:02.892200 30835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:02.893070 30835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:02.893121 30835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:02.893177 30835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:02.893206 30835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:02.901911 30835 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:46341
I20260806 18:33:02.901938 31044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:46341 every 8 connection(s)
I20260806 18:33:02.902971 30835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:33:02.910288 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 30835
I20260806 18:33:02.910739 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:45533
--local_ip_for_outbound_sockets=127.7.67.195
--tserver_master_addrs=127.7.67.254:44587
--webserver_port=38247
--webserver_interface=127.7.67.195
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:33:02 dist-test-slave-5bn2 krb5kdc[29623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041182, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:33:02.916350 30832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:02.904810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:55789 (local address 127.7.67.254:44587)
0806 18:33:02.904952 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:02.904956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:02.906067 (+ 1111us) server_negotiation.cc:407] Connection header received
0806 18:33:02.906989 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:02.906993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:02.907059 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:02.907138 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:02.908365 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:02.908898 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:02.910025 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:02.910271 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:02.910362 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:02.912847 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:02.912850 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:02.912852 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:02.912879 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:02.914294 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:02.914809 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:02.914813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:02.914815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:02.914929 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:02.915253 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:02.915256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:02.915258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:02.915448 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:02.915543 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:02.916037 (+ 494us) server_negotiation.cc:299] Negotiation successful
0806 18:33:02.916180 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:33:02.917176 31049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:02.905203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:55789)
0806 18:33:02.905903 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:02.905955 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:02.906768 (+ 813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:02.907337 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:02.907344 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:02.907706 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:02.908215 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:02.908229 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:02.909024 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:02.909028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:02.909897 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:02.909906 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:02.910422 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:02.910427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:02.910602 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:02.911226 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:33:02.911243 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:02.912671 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:02.914423 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:02.914430 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:02.914446 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:02.914710 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:02.915014 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:02.915018 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:02.915020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:02.915164 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:02.915556 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:02.915562 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:02.915809 (+ 247us) client_negotiation.cc:769] Sending connection context
0806 18:33:02.915987 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 18:33:02.916316 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":129,"threads_started":1}
I20260806 18:33:02.918334 31046 heartbeater.cc:347] Connected to a master server at 127.7.67.254:44587
I20260806 18:33:02.918624 31046 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:02.919304 31046 heartbeater.cc:511] Master 127.7.67.254:44587 requested a full tablet report, sending...
I20260806 18:33:02.920531 30534 ts_manager.cc:194] Registered new tserver with Master: 957e572b0e03411c9808169742a7aed6 (127.7.67.194:46341)
I20260806 18:33:02.921321 30534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:55789
W20260806 18:33:03.028904 31050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:03.029163 31050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:03.029204 31050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:03.033145 31050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:03.033203 31050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:03.033260 31050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:33:03.038439 31050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:45533
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=38247
--tserver_master_addrs=127.7.67.254:44587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:03.039469 31050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:03.040448 31050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:03.050379 31062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:03.050395 31058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:03.050478 31059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:03.050616 31050 server_base.cc:1034] running on GCE node
I20260806 18:33:03.051158 31050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:03.051793 31050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:03.052950 31050 hybrid_clock.cc:648] HybridClock initialized: now 1786041183052934 us; error 33 us; skew 500 ppm
Aug 06 18:33:03 dist-test-slave-5bn2 krb5kdc[29623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041183, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:03.055678 31050 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:33:03.056579 31050 webserver.cc:460] Webserver started at http://127.7.67.195:38247/ using document root <none> and password file <none>
I20260806 18:33:03.057140 31050 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:03.057221 31050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:03.061214 31050 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 18:33:03.063328 31075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:03.064527 31050 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:33:03.064641 31050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "a60bf7f6826245bf8fcd750e64815f22"
format_stamp: "Formatted at 2026-08-06 18:32:42 on dist-test-slave-5bn2"
server_key: "697495630b1f295187c6dacbfc8a18da"
server_key_iv: "3111ff4e293503468e1c940121d99279"
server_key_version: "encryptionkey@0"
I20260806 18:33:03.065181 31050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:03.080469 31050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:03.083588 31050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:03.083778 31050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:03.084424 31050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:03.085304 31050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:03.085355 31050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:03.085412 31050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:03.085440 31050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:03.094482 31050 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:45533
I20260806 18:33:03.094520 31246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:45533 every 8 connection(s)
I20260806 18:33:03.095521 31050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:33:03.098685 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 31050
Aug 06 18:33:03 dist-test-slave-5bn2 krb5kdc[29623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041183, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:33:03.108143 31254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:03.099483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43560 (local address 127.7.67.254:44587)
0806 18:33:03.099744 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:03.099747 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:03.099892 (+ 145us) server_negotiation.cc:407] Connection header received
0806 18:33:03.100021 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:03.100024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:03.100072 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:03.100163 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:03.100946 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.101611 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.102570 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.102937 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.103042 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:03.104512 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:03.104517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:03.104519 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:03.104544 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:03.106150 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:03.106903 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:03.106906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:03.106908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:03.107015 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:03.107277 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:03.107280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:03.107282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:03.107460 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:03.107660 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:03.107841 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 18:33:03.107961 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":69,"threads_started":1}
I20260806 18:33:03.109797 30832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:03.097347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:35431 (local address 127.7.67.254:44587)
0806 18:33:03.097491 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:03.097496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:03.098637 (+ 1141us) server_negotiation.cc:407] Connection header received
0806 18:33:03.099786 (+ 1149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:03.099790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:03.099873 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:03.099958 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:03.101293 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.101861 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.103275 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.103547 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.103663 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:03.106273 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:03.106276 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:03.106278 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:03.106301 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:03.107705 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:03.108249 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:03.108253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:03.108255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:03.108344 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:03.108692 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:03.108695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:03.108696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:03.108896 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:03.108994 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:03.109542 (+ 548us) server_negotiation.cc:299] Negotiation successful
0806 18:33:03.109654 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:33:03.110675 31252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:03.097760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:35431)
0806 18:33:03.098458 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:03.098517 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:03.099498 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:03.100179 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:03.100188 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:03.100567 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:03.101131 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:03.101151 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.102001 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.102004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:03.103137 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:03.103147 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.103712 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.103717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:03.103863 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:03.104355 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:33:03.104371 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:03.106104 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:03.107828 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:03.107834 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:03.107849 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:03.108161 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:03.108434 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:03.108437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:03.108439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:03.108593 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:03.109003 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:03.109010 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:03.109283 (+ 273us) client_negotiation.cc:769] Sending connection context
0806 18:33:03.109466 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 18:33:03.109743 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":108,"threads_started":1}
I20260806 18:33:03.111774 31250 heartbeater.cc:347] Connected to a master server at 127.7.67.254:44587
I20260806 18:33:03.112025 31250 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:03.112553 31250 heartbeater.cc:511] Master 127.7.67.254:44587 requested a full tablet report, sending...
I20260806 18:33:03.113567 30534 ts_manager.cc:194] Registered new tserver with Master: a60bf7f6826245bf8fcd750e64815f22 (127.7.67.195:45533)
I20260806 18:33:03.114120 30534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:35431
I20260806 18:33:03.121723 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:33:03.125829 30832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:03.122401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43574 (local address 127.7.67.254:44587)
0806 18:33:03.122548 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:03.122552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:03.122697 (+ 145us) server_negotiation.cc:407] Connection header received
0806 18:33:03.122873 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:03.122876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:03.122992 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:03.123089 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:33:03.123326 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.123845 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.124786 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.125007 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.125079 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:03.125352 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:33:03.125497 (+ 145us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 18:33:03.125705 (+ 208us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43574: token expired
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 18:33:03.125911 30832 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43574: token expired
W20260806 18:33:03.126231 30418 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.67.254:44587)
I20260806 18:33:03.132562 30832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:03.126537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43582 (local address 127.7.67.254:44587)
0806 18:33:03.126654 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:03.126657 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:03.126669 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:33:03.126747 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:03.126749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:03.126784 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:03.126841 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:03.127440 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.127889 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.128598 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.128781 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.128843 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:03.130029 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:03.130033 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:03.130035 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:03.130056 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:03.131307 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:03.131694 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:03.131696 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:03.131697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:03.131765 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:03.131947 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:03.131949 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:03.131950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:03.132105 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:03.132200 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:03.132341 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 18:33:03.132428 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:33:03.135071 30534 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:33:03.138382 30534 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:33:03.146536 30720 tablet_service.cc:1467] Processing CreateTablet for tablet 105ee5f841f04251917ff6e853e860ee (DEFAULT_TABLE table=test-table [id=72351f9090304e46bc747174dddfaed3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:03.147336 30720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105ee5f841f04251917ff6e853e860ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:03.149848 30653 tablet_bootstrap.cc:492] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1: Bootstrap starting.
I20260806 18:33:03.150817 30653 tablet_bootstrap.cc:654] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:03.151458 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:03.145040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:45533 (local address 127.0.0.1:41870)
0806 18:33:03.145159 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:03.145175 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:03.145251 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:03.146483 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:03.146486 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:03.146504 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:03.146617 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:03.146624 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.148122 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.148125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:03.149634 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:03.149647 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.151002 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.151006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:03.151112 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:03.151170 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 18:33:03.151225 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 18:33:03.151313 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":95}
I20260806 18:33:03.152154 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:03.145429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41870 (local address 127.7.67.195:45533)
0806 18:33:03.146085 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:03.146095 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:03.146114 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:33:03.146178 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:03.146182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:03.146331 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:03.146487 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:03.146713 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.148017 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.149780 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.150831 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.151654 (+ 823us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:03.151817 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:03.151875 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 18:33:03.151992 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":141,"threads_started":1}
I20260806 18:33:03.152278 31266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:03.145432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:46341 (local address 127.0.0.1:41276)
0806 18:33:03.145714 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:03.145728 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:03.145821 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:03.146697 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:03.146700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:03.146719 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:03.146809 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:03.146815 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.148167 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.148170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:03.149635 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:03.149647 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.151917 (+ 2270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.151920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:03.152010 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:03.152035 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:33:03.152086 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 18:33:03.152157 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":128,"thread_start_us":141,"threads_started":1}
I20260806 18:33:03.152299 30653 tablet_bootstrap.cc:492] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1: No bootstrap required, opened a new log
I20260806 18:33:03.152391 30653 ts_tablet_manager.cc:1397] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 18:33:03.152843 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:03.145628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41276 (local address 127.7.67.194:46341)
0806 18:33:03.145968 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:03.145987 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:03.146016 (+ 29us) server_negotiation.cc:407] Connection header received
0806 18:33:03.146101 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:03.146107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:03.146335 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:03.146489 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:03.146968 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.148031 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.150947 (+ 2916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.151779 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.152453 (+ 674us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:03.152614 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:03.152658 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 18:33:03.152731 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":149,"threads_started":1}
I20260806 18:33:03.152870 30653 raft_consensus.cc:348] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:33:03.153000 30653 raft_consensus.cc:374] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:03.153049 30653 raft_consensus.cc:729] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 251302a9dffd4ce8a7c1330df743f0c1, State: Initialized, Role: FOLLOWER
I20260806 18:33:03.153187 30653 consensus_queue.cc:260] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:33:03.153463 30653 ts_tablet_manager.cc:1428] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:33:03.155643 31146 tablet_service.cc:1467] Processing CreateTablet for tablet 105ee5f841f04251917ff6e853e860ee (DEFAULT_TABLE table=test-table [id=72351f9090304e46bc747174dddfaed3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:03.156383 30946 tablet_service.cc:1467] Processing CreateTablet for tablet 105ee5f841f04251917ff6e853e860ee (DEFAULT_TABLE table=test-table [id=72351f9090304e46bc747174dddfaed3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:03.156868 31146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105ee5f841f04251917ff6e853e860ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:03.157437 30946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105ee5f841f04251917ff6e853e860ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:03.165328 31272 tablet_bootstrap.cc:492] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6: Bootstrap starting.
I20260806 18:33:03.165833 31275 tablet_bootstrap.cc:492] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22: Bootstrap starting.
I20260806 18:33:03.167981 31275 tablet_bootstrap.cc:654] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:03.168292 31272 tablet_bootstrap.cc:654] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:03.169253 31275 log.cc:824] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:03.169253 31272 log.cc:824] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:03.171499 31272 tablet_bootstrap.cc:492] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6: No bootstrap required, opened a new log
I20260806 18:33:03.171499 31275 tablet_bootstrap.cc:492] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22: No bootstrap required, opened a new log
I20260806 18:33:03.171711 31272 ts_tablet_manager.cc:1397] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 18:33:03.171729 31275 ts_tablet_manager.cc:1397] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:33:03.175721 31275 raft_consensus.cc:348] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:33:03.175721 31272 raft_consensus.cc:348] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:33:03.176033 31275 raft_consensus.cc:374] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:03.176033 31272 raft_consensus.cc:374] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:03.176095 31275 raft_consensus.cc:729] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a60bf7f6826245bf8fcd750e64815f22, State: Initialized, Role: FOLLOWER
I20260806 18:33:03.176095 31272 raft_consensus.cc:729] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 957e572b0e03411c9808169742a7aed6, State: Initialized, Role: FOLLOWER
I20260806 18:33:03.176582 31275 consensus_queue.cc:260] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:33:03.176582 31272 consensus_queue.cc:260] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:33:03.177306 31250 heartbeater.cc:503] Master 127.7.67.254:44587 was elected leader, sending a full tablet report...
I20260806 18:33:03.177675 31275 ts_tablet_manager.cc:1428] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:33:03.177675 31272 ts_tablet_manager.cc:1428] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:33:03.177898 31046 heartbeater.cc:503] Master 127.7.67.254:44587 was elected leader, sending a full tablet report...
W20260806 18:33:03.205845 30828 tablet.cc:2369] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:33:03.231277 31281 raft_consensus.cc:482] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:33:03.231503 31281 raft_consensus.cc:504] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:33:03.232708 31281 leader_election.cc:288] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a60bf7f6826245bf8fcd750e64815f22 (127.7.67.195:45533), 251302a9dffd4ce8a7c1330df743f0c1 (127.7.67.193:36471)
I20260806 18:33:03.237458 30841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:03.233133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:48953 (local address 127.7.67.193:36471)
0806 18:33:03.233260 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:03.233265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:03.233282 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:33:03.233346 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:03.233349 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:03.233413 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:03.233511 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:03.233799 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.234598 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.236238 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.236888 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.237000 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:03.237132 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:03.237248 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 18:33:03.237319 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:33:03.237458 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:03.233221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:52349 (local address 127.7.67.195:45533)
0806 18:33:03.233337 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:03.233341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:03.233550 (+ 209us) server_negotiation.cc:407] Connection header received
0806 18:33:03.233657 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:03.233659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:03.233699 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:03.233763 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:03.233988 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.234687 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.236236 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.236830 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.236951 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:03.237016 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:03.237250 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 18:33:03.237319 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:33:03.237493 31049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:03.232970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:36471 (local address 127.7.67.194:48953)
0806 18:33:03.233128 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:03.233145 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:03.233254 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:03.233547 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:03.233550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:03.233564 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:03.233689 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:03.233695 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.234720 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.234722 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:03.236114 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:03.236122 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.236982 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.236986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:03.237150 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:03.237167 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 18:33:03.237230 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 18:33:03.237320 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":120}
I20260806 18:33:03.237501 31283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:03.232970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:45533 (local address 127.7.67.194:52349)
0806 18:33:03.233454 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:03.233465 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:03.233559 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:03.233786 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:03.233789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:03.233807 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:03.233896 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:03.233900 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.234781 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.234784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:03.236110 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:03.236122 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.236957 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.236961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:03.237102 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:03.237143 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 18:33:03.237225 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 18:33:03.237320 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":133,"thread_start_us":112,"threads_started":1}
I20260806 18:33:03.238205 30754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105ee5f841f04251917ff6e853e860ee" candidate_uuid: "957e572b0e03411c9808169742a7aed6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "251302a9dffd4ce8a7c1330df743f0c1" is_pre_election: true
I20260806 18:33:03.238309 31179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105ee5f841f04251917ff6e853e860ee" candidate_uuid: "957e572b0e03411c9808169742a7aed6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a60bf7f6826245bf8fcd750e64815f22" is_pre_election: true
I20260806 18:33:03.238554 31179 raft_consensus.cc:2454] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 957e572b0e03411c9808169742a7aed6 in term 0.
I20260806 18:33:03.238555 30754 raft_consensus.cc:2454] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 957e572b0e03411c9808169742a7aed6 in term 0.
I20260806 18:33:03.239058 30870 leader_election.cc:302] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 957e572b0e03411c9808169742a7aed6, a60bf7f6826245bf8fcd750e64815f22; no voters:
I20260806 18:33:03.239352 31281 raft_consensus.cc:2781] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:33:03.239441 31281 raft_consensus.cc:482] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:33:03.239493 31281 raft_consensus.cc:3037] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:03.240576 31281 raft_consensus.cc:504] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:33:03.240931 31281 leader_election.cc:288] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [CANDIDATE]: Term 1 election: Requested vote from peers a60bf7f6826245bf8fcd750e64815f22 (127.7.67.195:45533), 251302a9dffd4ce8a7c1330df743f0c1 (127.7.67.193:36471)
I20260806 18:33:03.241256 31179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105ee5f841f04251917ff6e853e860ee" candidate_uuid: "957e572b0e03411c9808169742a7aed6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a60bf7f6826245bf8fcd750e64815f22"
I20260806 18:33:03.241384 31179 raft_consensus.cc:3037] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:03.241549 30754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105ee5f841f04251917ff6e853e860ee" candidate_uuid: "957e572b0e03411c9808169742a7aed6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "251302a9dffd4ce8a7c1330df743f0c1"
I20260806 18:33:03.241715 30754 raft_consensus.cc:3037] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:03.242720 31179 raft_consensus.cc:2454] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 957e572b0e03411c9808169742a7aed6 in term 1.
I20260806 18:33:03.242945 30754 raft_consensus.cc:2454] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 957e572b0e03411c9808169742a7aed6 in term 1.
I20260806 18:33:03.243266 30865 leader_election.cc:302] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 251302a9dffd4ce8a7c1330df743f0c1, 957e572b0e03411c9808169742a7aed6; no voters:
I20260806 18:33:03.243446 31281 raft_consensus.cc:2781] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:33:03.243687 31281 raft_consensus.cc:686] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [term 1 LEADER]: Becoming Leader. State: Replica: 957e572b0e03411c9808169742a7aed6, State: Running, Role: LEADER
I20260806 18:33:03.244086 31281 consensus_queue.cc:237] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:33:03.247294 30532 catalog_manager.cc:5495] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 957e572b0e03411c9808169742a7aed6 (127.7.67.194). New cstate: current_term: 1 leader_uuid: "957e572b0e03411c9808169742a7aed6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } health_report { overall_health: UNKNOWN } } }
I20260806 18:33:03.275337 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:03.271833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41288 (local address 127.7.67.194:46341)
0806 18:33:03.271989 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:03.271994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:03.272009 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:33:03.272127 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:03.272129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:03.272195 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:03.272306 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:33:03.272540 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.273083 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.273902 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:03.274119 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:03.274204 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:03.274497 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:33:03.274940 (+ 443us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:33:03.275059 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:03.275117 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 18:33:03.275174 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:33:03.281102 31179 raft_consensus.cc:1261] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22 [term 1 FOLLOWER]: Refusing update from remote peer 957e572b0e03411c9808169742a7aed6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:33:03.281100 30754 raft_consensus.cc:1261] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Refusing update from remote peer 957e572b0e03411c9808169742a7aed6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:33:03.281771 31284 consensus_queue.cc:1035] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:33:03.281992 31281 consensus_queue.cc:1035] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:33:03.288863 31286 mvcc.cc:205] Tried to move back new op lower bound from 7315624686712676352 to 7315624686570479616. Current Snapshot: MvccSnapshot[applied={T|T < 7315624686712676352}]
I20260806 18:33:03.289004 31287 mvcc.cc:205] Tried to move back new op lower bound from 7315624686712676352 to 7315624686570479616. Current Snapshot: MvccSnapshot[applied={T|T < 7315624686712676352}]
I20260806 18:33:03.290553 31288 mvcc.cc:205] Tried to move back new op lower bound from 7315624686712676352 to 7315624686570479616. Current Snapshot: MvccSnapshot[applied={T|T < 7315624686712676352 or (T in {7315624686712676352})}]
I20260806 18:33:03.298202 30532 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:33:03.298359 30532 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:33:03.300634 30532 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3: Sending DeleteTablet for 3 replicas of tablet 105ee5f841f04251917ff6e853e860ee
I20260806 18:33:03.301311 30946 tablet_service.cc:1514] Processing DeleteTablet for tablet 105ee5f841f04251917ff6e853e860ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:03 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:41276
I20260806 18:33:03.301313 31146 tablet_service.cc:1514] Processing DeleteTablet for tablet 105ee5f841f04251917ff6e853e860ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:03 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:41870
I20260806 18:33:03.301578 30720 tablet_service.cc:1514] Processing DeleteTablet for tablet 105ee5f841f04251917ff6e853e860ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:03 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:48498
I20260806 18:33:03.301957 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 30587
I20260806 18:33:03.301960 31302 tablet_replica.cc:330] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22: stopping tablet replica
I20260806 18:33:03.302142 31303 tablet_replica.cc:330] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1: stopping tablet replica
I20260806 18:33:03.302271 31302 raft_consensus.cc:2229] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:33:03.302596 31302 raft_consensus.cc:2258] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:33:03.303028 31301 tablet_replica.cc:330] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6: stopping tablet replica
I20260806 18:33:03.303344 31301 raft_consensus.cc:2229] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:33:03.303762 31301 raft_consensus.cc:2258] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:33:03.317480 31302 ts_tablet_manager.cc:1905] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:33:03.317724 31301 ts_tablet_manager.cc:1905] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:33:03.320375 31302 ts_tablet_manager.cc:1918] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:33:03.320526 31302 log.cc:1196] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/105ee5f841f04251917ff6e853e860ee
I20260806 18:33:03.320698 31301 ts_tablet_manager.cc:1918] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:33:03.320883 31302 ts_tablet_manager.cc:1939] T 105ee5f841f04251917ff6e853e860ee P a60bf7f6826245bf8fcd750e64815f22: Deleting consensus metadata
I20260806 18:33:03.320876 31301 log.cc:1196] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/105ee5f841f04251917ff6e853e860ee
I20260806 18:33:03.321224 31301 ts_tablet_manager.cc:1939] T 105ee5f841f04251917ff6e853e860ee P 957e572b0e03411c9808169742a7aed6: Deleting consensus metadata
I20260806 18:33:03.322454 30498 catalog_manager.cc:4841] TS 957e572b0e03411c9808169742a7aed6 (127.7.67.194:46341): tablet 105ee5f841f04251917ff6e853e860ee (table test-table [id=72351f9090304e46bc747174dddfaed3]) successfully deleted
I20260806 18:33:03.322501 30514 catalog_manager.cc:4841] TS a60bf7f6826245bf8fcd750e64815f22 (127.7.67.195:45533): tablet 105ee5f841f04251917ff6e853e860ee (table test-table [id=72351f9090304e46bc747174dddfaed3]) successfully deleted
W20260806 18:33:03.324237 30507 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.193:36471 (error 108)
I20260806 18:33:03.324422 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 30835
I20260806 18:33:03.329548 31266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:03.329057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:36471 (local address 127.0.0.1:48504)
0806 18:33:03.329297 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:03.329399 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:36471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
W20260806 18:33:03.329857 30507 catalog_manager.cc:4584] TS 251302a9dffd4ce8a7c1330df743f0c1 (127.7.67.193:36471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 105ee5f841f04251917ff6e853e860ee: Network error: Client connection negotiation failed: client connection to 127.7.67.193:36471: connect: Connection refused (error 111)
I20260806 18:33:03.341262 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 31050
I20260806 18:33:03.351984 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 30475
I20260806 18:33:23.366921 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:44587
--webserver_interface=127.7.67.254
--webserver_port=45789
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:44587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:33:23.485601 31307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:23.485872 31307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:23.485919 31307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:23.489979 31307 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 18:33:23.490038 31307 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 18:33:23.490052 31307 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 18:33:23.490065 31307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:33:23.490100 31307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:23.490123 31307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:23.490154 31307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:33:23.490178 31307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:33:23.495841 31307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:44587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:44587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=45789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:23.497184 31307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:23.498265 31307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:23.507270 31314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:23.507313 31313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:23.507366 31307 server_base.cc:1034] running on GCE node
W20260806 18:33:23.507296 31317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:23.508123 31307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:23.509286 31307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:23.510450 31307 hybrid_clock.cc:648] HybridClock initialized: now 1786041203510437 us; error 27 us; skew 500 ppm
Aug 06 18:33:23 dist-test-slave-5bn2 krb5kdc[29623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041203, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:23.517885 31307 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:33:23.518960 31307 webserver.cc:460] Webserver started at http://127.7.67.254:45789/ using document root <none> and password file <none>
I20260806 18:33:23.519565 31307 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:23.519634 31307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:23.523947 31307 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:33:23.526369 31330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:23.527778 31307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:33:23.527904 31307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "a492c393d808401285fe57712a2ecaa3"
format_stamp: "Formatted at 2026-08-06 18:32:41 on dist-test-slave-5bn2"
server_key: "030e90a291d1d7ce1a63064301b285a3"
server_key_iv: "358ae873d13bac407c122f8677e94f23"
server_key_version: "encryptionkey@0"
I20260806 18:33:23.528537 31307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:23.545746 31307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:23.548887 31307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:23.549075 31307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:23.556288 31307 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:44587
I20260806 18:33:23.556600 31410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:44587 every 8 connection(s)
I20260806 18:33:23.557459 31307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:33:23.562286 31412 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:33:23.564208 31412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3: Bootstrap starting.
I20260806 18:33:23.565429 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 31307
I20260806 18:33:23.565884 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:36471
--local_ip_for_outbound_sockets=127.7.67.193
--tserver_master_addrs=127.7.67.254:44587
--webserver_port=34007
--webserver_interface=127.7.67.193
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:33:23.582803 31412 log.cc:824] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:23.594400 31412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:33:23.594797 31412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3: Bootstrap complete.
I20260806 18:33:23.598886 31412 raft_consensus.cc:348] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a492c393d808401285fe57712a2ecaa3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44587 } }
I20260806 18:33:23.599319 31412 raft_consensus.cc:729] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a492c393d808401285fe57712a2ecaa3, State: Initialized, Role: FOLLOWER
I20260806 18:33:23.599896 31412 consensus_queue.cc:260] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a492c393d808401285fe57712a2ecaa3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44587 } }
I20260806 18:33:23.600023 31412 raft_consensus.cc:388] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:33:23.600087 31412 raft_consensus.cc:482] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:33:23.600185 31412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 2 FOLLOWER]: Advancing to term 3
I20260806 18:33:23.602136 31412 raft_consensus.cc:504] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a492c393d808401285fe57712a2ecaa3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44587 } }
I20260806 18:33:23.602458 31412 leader_election.cc:302] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a492c393d808401285fe57712a2ecaa3; no voters:
I20260806 18:33:23.602784 31412 leader_election.cc:288] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 18:33:23.602923 31417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 18:33:23.603145 31417 raft_consensus.cc:686] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [term 3 LEADER]: Becoming Leader. State: Replica: a492c393d808401285fe57712a2ecaa3, State: Running, Role: LEADER
I20260806 18:33:23.603526 31417 consensus_queue.cc:237] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a492c393d808401285fe57712a2ecaa3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44587 } }
I20260806 18:33:23.604141 31412 sys_catalog.cc:564] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:33:23.604517 31417 sys_catalog.cc:455] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a492c393d808401285fe57712a2ecaa3. Latest consensus state: current_term: 3 leader_uuid: "a492c393d808401285fe57712a2ecaa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a492c393d808401285fe57712a2ecaa3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44587 } } }
I20260806 18:33:23.604719 31417 sys_catalog.cc:458] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:23.605165 31421 sys_catalog.cc:455] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a492c393d808401285fe57712a2ecaa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a492c393d808401285fe57712a2ecaa3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44587 } } }
I20260806 18:33:23.605278 31421 sys_catalog.cc:458] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3 [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:23.605527 31424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:33:23.611382 31424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:33:23.612223 31424 catalog_manager.cc:1259] Loaded cluster ID: e2bb8d45ea584d28bd27dce4196efb07
I20260806 18:33:23.612304 31424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:33:23.615010 31424 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:33:23.615839 31424 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3: Loaded TSK: 0
I20260806 18:33:23.615939 31424 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a492c393d808401285fe57712a2ecaa3: Loaded TSK: 1
F20260806 18:33:23.616741 31424 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff3f2dcd9ad google::LogMessage::Fail() at ??:0
@ 0x7ff3f2dd088a google::LogMessage::SendToLog() at ??:0
@ 0x7ff3f2dcd4de google::LogMessage::Flush() at ??:0
@ 0x7ff3f2dcea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff3f5a9118d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7ff3f5a89f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7ff3fd7327bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7ff3fd70c99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7ff3fd70d178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7ff3fd74edf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7ff3fd76d82b std::function<>::operator()() at ??:0
@ 0x7ff3fd70cb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7ff3fd70df3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff3fd70a579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7ff3fd74e640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ff3f501f3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff3f501fc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7ff3f50213f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ff3f5010512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff3f66546db start_thread at ??:0
@ 0x7ff3f18cb61f clone at ??:0
W20260806 18:33:23.707734 31415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:23.707995 31415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:23.708039 31415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:23.712070 31415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:23.712131 31415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:23.712196 31415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:33:23.717620 31415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:36471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=34007
--tserver_master_addrs=127.7.67.254:44587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:23.718698 31415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:23.719751 31415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:23.729540 31495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:23.729523 31494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:23.729581 31497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:23.730086 31415 server_base.cc:1034] running on GCE node
I20260806 18:33:23.730487 31415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:23.731140 31415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:23.732322 31415 hybrid_clock.cc:648] HybridClock initialized: now 1786041203732300 us; error 40 us; skew 500 ppm
Aug 06 18:33:23 dist-test-slave-5bn2 krb5kdc[29623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041203, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:23.735148 31415 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:33:23.736053 31415 webserver.cc:460] Webserver started at http://127.7.67.193:34007/ using document root <none> and password file <none>
I20260806 18:33:23.736615 31415 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:23.736678 31415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:23.740649 31415 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 18:33:23.742904 31506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:23.744165 31415 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:33:23.744269 31415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "251302a9dffd4ce8a7c1330df743f0c1"
format_stamp: "Formatted at 2026-08-06 18:32:41 on dist-test-slave-5bn2"
server_key: "f07c8e03ed5e47f65450d5d38f36f6fa"
server_key_iv: "f5a611975e8fa279bb10bb0c42d9ae2f"
server_key_version: "encryptionkey@0"
I20260806 18:33:23.744825 31415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:23.756457 31415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:23.759572 31415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:23.759743 31415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:23.760340 31415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:23.761435 31520 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:33:23.765581 31415 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:33:23.765640 31415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260806 18:33:23.765688 31415 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:33:23.767433 31415 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:33:23.767494 31415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260806 18:33:23.767776 31520 tablet_bootstrap.cc:492] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1: Bootstrap starting.
I20260806 18:33:23.778321 31415 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:36471
I20260806 18:33:23.778465 31672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:36471 every 8 connection(s)
I20260806 18:33:23.779773 31415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:33:23.784080 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 31415
I20260806 18:33:23.784633 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:46341
--local_ip_for_outbound_sockets=127.7.67.194
--tserver_master_addrs=127.7.67.254:44587
--webserver_port=42389
--webserver_interface=127.7.67.194
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:33:23.789230 31520 log.cc:824] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:23.796252 31520 tablet_bootstrap.cc:492] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:33:23.796610 31520 tablet_bootstrap.cc:492] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1: Bootstrap complete.
I20260806 18:33:23.797626 31520 ts_tablet_manager.cc:1397] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1: Time spent bootstrapping tablet: real 0.030s user 0.018s sys 0.009s
I20260806 18:33:23.800067 31520 raft_consensus.cc:348] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:33:23.800469 31520 raft_consensus.cc:729] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 251302a9dffd4ce8a7c1330df743f0c1, State: Initialized, Role: FOLLOWER
I20260806 18:33:23.800951 31520 consensus_queue.cc:260] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:33:23.801896 31520 ts_tablet_manager.cc:1428] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1: Time spent starting tablet: real 0.004s user 0.003s sys 0.004s
W20260806 18:33:23.873807 31676 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:33:23.782126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:39523)
0806 18:33:23.782769 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:23.782834 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:23.783982 (+ 1148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:23.872768 (+ 88786us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":178,"threads_started":1}
W20260806 18:33:23.874354 31673 heartbeater.cc:644] Failed to heartbeat to 127.7.67.254:44587 (0 consecutive failures): Network error: Failed to ping master at 127.7.67.254:44587: Client connection negotiation failed: client connection to 127.7.67.254:44587: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 18:33:23.875125 31676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:23.874822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:60683)
0806 18:33:23.874982 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:23.875057 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 18:33:23.875779 31676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:23.875574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:32807)
0806 18:33:23.875674 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:23.875723 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 18:33:23.875962 31673 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:33:23.927897 31677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:23.928174 31677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:23.928218 31677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:23.932273 31677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:23.932339 31677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:23.932399 31677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:33:23.937835 31677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:46341
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=42389
--tserver_master_addrs=127.7.67.254:44587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:23.938935 31677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:23.939929 31677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:23.949707 31690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:23.949733 31689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:23.949707 31693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:23.950006 31677 server_base.cc:1034] running on GCE node
I20260806 18:33:23.950450 31677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:23.951139 31677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:23.952297 31677 hybrid_clock.cc:648] HybridClock initialized: now 1786041203952277 us; error 34 us; skew 500 ppm
Aug 06 18:33:23 dist-test-slave-5bn2 krb5kdc[29623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041203, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:23.955072 31677 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:33:23.955965 31677 webserver.cc:460] Webserver started at http://127.7.67.194:42389/ using document root <none> and password file <none>
I20260806 18:33:23.956518 31677 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:23.956588 31677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:23.960438 31677 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:33:23.962518 31701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:23.963788 31677 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:33:23.963917 31677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "957e572b0e03411c9808169742a7aed6"
format_stamp: "Formatted at 2026-08-06 18:32:41 on dist-test-slave-5bn2"
server_key: "b18369095d56f9705b9d7ae540d0fcba"
server_key_iv: "64d5615c0c2c17a054b74ae889b4cd59"
server_key_version: "encryptionkey@0"
I20260806 18:33:23.964474 31677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:24.007397 31677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:24.010680 31677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:24.010918 31677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:24.011575 31677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:24.012457 31677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:24.012509 31677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:24.012565 31677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:24.012594 31677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:24.021514 31677 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:46341
I20260806 18:33:24.021560 31874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:46341 every 8 connection(s)
I20260806 18:33:24.022578 31677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:33:24.025509 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 31677
I20260806 18:33:24.026002 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:45533
--local_ip_for_outbound_sockets=127.7.67.195
--tserver_master_addrs=127.7.67.254:44587
--webserver_port=38247
--webserver_interface=127.7.67.195
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:33:24.026708 31879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:24.024831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:53169)
0806 18:33:24.025619 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:24.025762 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":124,"threads_started":1}
W20260806 18:33:24.027207 31876 heartbeater.cc:644] Failed to heartbeat to 127.7.67.254:44587 (0 consecutive failures): Network error: Failed to ping master at 127.7.67.254:44587: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
I20260806 18:33:24.027864 31879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:24.027593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:54083)
0806 18:33:24.027720 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:24.027782 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 18:33:24.028455 31879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:24.028256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:60277)
0806 18:33:24.028358 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:24.028405 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 18:33:24.028605 31876 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:33:24.141634 31881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:24.141968 31881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:24.142050 31881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:24.146130 31881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:24.146210 31881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:24.146307 31881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:33:24.151857 31881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:45533
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=38247
--tserver_master_addrs=127.7.67.254:44587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:24.153039 31881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:24.154101 31881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:24.163967 31895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:24.163967 31888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:24.164001 31897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:24.164227 31881 server_base.cc:1034] running on GCE node
I20260806 18:33:24.164736 31881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:24.165377 31881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:24.166539 31881 hybrid_clock.cc:648] HybridClock initialized: now 1786041204166519 us; error 36 us; skew 500 ppm
Aug 06 18:33:24 dist-test-slave-5bn2 krb5kdc[29623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041204, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:24.169399 31881 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:33:24.170248 31881 webserver.cc:460] Webserver started at http://127.7.67.195:38247/ using document root <none> and password file <none>
I20260806 18:33:24.170802 31881 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:24.170904 31881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:24.174593 31881 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 18:33:24.176718 31904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:24.177968 31881 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:33:24.178086 31881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "a60bf7f6826245bf8fcd750e64815f22"
format_stamp: "Formatted at 2026-08-06 18:32:42 on dist-test-slave-5bn2"
server_key: "697495630b1f295187c6dacbfc8a18da"
server_key_iv: "3111ff4e293503468e1c940121d99279"
server_key_version: "encryptionkey@0"
I20260806 18:33:24.178625 31881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:24.215265 31881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:24.218182 31881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:24.218376 31881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:24.219026 31881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:24.219904 31881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:24.219952 31881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:24.220010 31881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:24.220038 31881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:24.228859 31881 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:45533
I20260806 18:33:24.228888 32089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:45533 every 8 connection(s)
I20260806 18:33:24.229869 31881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:33:24.233498 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:24.232014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:37925)
0806 18:33:24.232666 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:24.232775 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":115,"threads_started":1}
W20260806 18:33:24.233999 32090 heartbeater.cc:644] Failed to heartbeat to 127.7.67.254:44587 (0 consecutive failures): Network error: Failed to ping master at 127.7.67.254:44587: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
I20260806 18:33:24.234340 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 31881
I20260806 18:33:24.234654 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:24.234413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:56521)
0806 18:33:24.234539 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:24.234595 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 18:33:24.235381 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:24.235122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:42741)
0806 18:33:24.235262 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:24.235316 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 18:33:24.235584 32090 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 18:33:24.877101 32094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:24.876617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:42455)
0806 18:33:24.876912 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:24.876979 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":77,"threads_started":1}
I20260806 18:33:25.029651 32095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:25.029190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:40591)
0806 18:33:25.029480 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:25.029541 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":73,"threads_started":1}
I20260806 18:33:25.236589 32096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:25.236113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:50439)
0806 18:33:25.236386 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:25.236471 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":72,"threads_started":1}
I20260806 18:33:25.529556 32099 raft_consensus.cc:482] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:33:25.529778 32099 raft_consensus.cc:504] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:33:25.531153 32099 leader_election.cc:288] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a60bf7f6826245bf8fcd750e64815f22 (127.7.67.195:45533), 957e572b0e03411c9808169742a7aed6 (127.7.67.194:46341)
Aug 06 18:33:25 dist-test-slave-5bn2 krb5kdc[29623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041203, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.195@KRBTEST.COM
Aug 06 18:33:25 dist-test-slave-5bn2 krb5kdc[29623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041203, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.194@KRBTEST.COM
I20260806 18:33:25.545248 32100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:25.531341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:45533 (local address 127.7.67.193:33665)
0806 18:33:25.531644 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:25.531659 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:25.531772 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:25.533437 (+ 1665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:25.533442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:25.534032 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:25.534535 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:25.534553 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:25.535964 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:25.535967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:25.536948 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:25.536960 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:25.537753 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:25.537757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:25.537930 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:25.538549 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:33:25.538572 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:25.540215 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:25.542678 (+ 2463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:25.542684 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:25.542695 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:25.543016 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:25.543403 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:25.543407 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:25.543409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:25.543592 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:25.544260 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:25.544266 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:25.544513 (+ 247us) client_negotiation.cc:769] Sending connection context
0806 18:33:25.544685 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 18:33:25.545037 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":198,"spinlock_wait_cycles":28288,"thread_start_us":65,"threads_started":1}
I20260806 18:33:25.545264 32104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:25.531665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:57853 (local address 127.7.67.194:46341)
0806 18:33:25.532010 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:25.532021 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:25.532054 (+ 33us) server_negotiation.cc:407] Connection header received
0806 18:33:25.532714 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:25.532720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:25.533067 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:25.533666 (+ 599us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:25.534770 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:25.535826 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:25.537066 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:25.537618 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:25.537769 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:25.540713 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:25.540719 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:25.540734 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:25.540769 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:25.542545 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:25.543251 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:25.543256 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:25.543260 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:25.543389 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:25.543706 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:25.543708 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:25.543710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:25.544138 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:25.544260 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:25.544822 (+ 562us) server_negotiation.cc:299] Negotiation successful
0806 18:33:25.545097 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":118,"threads_started":1}
I20260806 18:33:25.545282 32103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:25.531493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:33665 (local address 127.7.67.195:45533)
0806 18:33:25.531978 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:25.532001 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:25.532050 (+ 49us) server_negotiation.cc:407] Connection header received
0806 18:33:25.532728 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:25.532733 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:25.533073 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:25.533354 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:25.534790 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:25.535808 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:25.537074 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:25.537620 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:25.537784 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:25.540491 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:25.540497 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:25.540511 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:25.540542 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:25.542545 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:25.543142 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:25.543148 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:25.543154 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:25.543306 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:25.543706 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:25.543710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:25.543712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:25.544139 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:25.544266 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:25.544822 (+ 556us) server_negotiation.cc:299] Negotiation successful
0806 18:33:25.545086 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":103,"threads_started":1}
I20260806 18:33:25.545275 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:25.531416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:46341 (local address 127.7.67.193:57853)
0806 18:33:25.531734 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:25.531749 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:25.531862 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:25.533609 (+ 1747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:25.533613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:25.534031 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:25.534536 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:25.534586 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:25.535960 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:25.535963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:25.536948 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:25.536959 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:25.537740 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:25.537744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:25.537932 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:25.538662 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:33:25.538677 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:25.540514 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:25.542933 (+ 2419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:25.542936 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:25.542937 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:25.543147 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:25.543513 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:25.543515 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:25.543517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:25.543606 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:25.544259 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:25.544265 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:25.544512 (+ 247us) client_negotiation.cc:769] Sending connection context
0806 18:33:25.544685 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 18:33:25.545115 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":270,"spinlock_wait_cycles":14720,"thread_start_us":74,"threads_started":1}
W20260806 18:33:25.548121 31512 leader_election.cc:341] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a60bf7f6826245bf8fcd750e64815f22 (127.7.67.195:45533): Not found: Tablet not found: 105ee5f841f04251917ff6e853e860ee
W20260806 18:33:25.548300 31507 leader_election.cc:341] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 957e572b0e03411c9808169742a7aed6 (127.7.67.194:46341): Not found: Tablet not found: 105ee5f841f04251917ff6e853e860ee
I20260806 18:33:25.548434 31507 leader_election.cc:302] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 251302a9dffd4ce8a7c1330df743f0c1; no voters: 957e572b0e03411c9808169742a7aed6, a60bf7f6826245bf8fcd750e64815f22
I20260806 18:33:25.548727 32099 raft_consensus.cc:2726] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:33:25.878140 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:25.877865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:52589)
0806 18:33:25.877989 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:25.878046 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 18:33:26.030993 32107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:26.030522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:55243)
0806 18:33:26.030788 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:26.030873 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":100,"threads_started":1}
I20260806 18:33:26.237908 32110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:26.237418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:55637)
0806 18:33:26.237724 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:26.237795 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":96,"threads_started":1}
I20260806 18:33:26.879462 32111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:26.879038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:41469)
0806 18:33:26.879292 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:26.879353 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":79,"threads_started":1}
I20260806 18:33:27.032171 32115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:27.031720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:60393)
0806 18:33:27.032002 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:27.032065 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":93,"threads_started":1}
I20260806 18:33:27.239192 32116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:27.238696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:42739)
0806 18:33:27.239011 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:27.239072 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":76,"threads_started":1}
I20260806 18:33:27.275607 32117 raft_consensus.cc:482] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:33:27.275784 32117 raft_consensus.cc:504] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:33:27.276206 32117 leader_election.cc:288] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a60bf7f6826245bf8fcd750e64815f22 (127.7.67.195:45533), 957e572b0e03411c9808169742a7aed6 (127.7.67.194:46341)
W20260806 18:33:27.277081 31507 leader_election.cc:341] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 957e572b0e03411c9808169742a7aed6 (127.7.67.194:46341): Not found: Tablet not found: 105ee5f841f04251917ff6e853e860ee
W20260806 18:33:27.277223 31512 leader_election.cc:341] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a60bf7f6826245bf8fcd750e64815f22 (127.7.67.195:45533): Not found: Tablet not found: 105ee5f841f04251917ff6e853e860ee
I20260806 18:33:27.277321 31512 leader_election.cc:302] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 251302a9dffd4ce8a7c1330df743f0c1; no voters: 957e572b0e03411c9808169742a7aed6, a60bf7f6826245bf8fcd750e64815f22
I20260806 18:33:27.277487 32117 raft_consensus.cc:2726] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:33:27.884986 32118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:27.880359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:60395)
0806 18:33:27.884785 (+ 4426us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:27.884860 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4322,"thread_start_us":113,"threads_started":1}
I20260806 18:33:28.033322 32119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:28.032898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:33605)
0806 18:33:28.033170 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:28.033226 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":72,"threads_started":1}
I20260806 18:33:28.240485 32121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:28.240031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:39203)
0806 18:33:28.240314 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:28.240385 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":89,"threads_started":1}
I20260806 18:33:28.887897 32122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:28.887386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:53291)
0806 18:33:28.887726 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:28.887790 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":98,"threads_started":1}
I20260806 18:33:29.034720 32123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:29.034226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:50643)
0806 18:33:29.034545 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:29.034613 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":97,"threads_started":1}
I20260806 18:33:29.241890 32125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:29.241366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:51131)
0806 18:33:29.241690 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:29.241771 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":114,"threads_started":1}
I20260806 18:33:29.889092 32126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:29.888655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:35487)
0806 18:33:29.888924 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:29.888988 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":81,"threads_started":1}
I20260806 18:33:29.986034 32133 raft_consensus.cc:482] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:33:29.986171 32133 raft_consensus.cc:504] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:33:29.986596 32133 leader_election.cc:288] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a60bf7f6826245bf8fcd750e64815f22 (127.7.67.195:45533), 957e572b0e03411c9808169742a7aed6 (127.7.67.194:46341)
W20260806 18:33:29.987372 31507 leader_election.cc:341] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 957e572b0e03411c9808169742a7aed6 (127.7.67.194:46341): Not found: Tablet not found: 105ee5f841f04251917ff6e853e860ee
W20260806 18:33:29.987486 31512 leader_election.cc:341] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a60bf7f6826245bf8fcd750e64815f22 (127.7.67.195:45533): Not found: Tablet not found: 105ee5f841f04251917ff6e853e860ee
I20260806 18:33:29.987581 31512 leader_election.cc:302] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 251302a9dffd4ce8a7c1330df743f0c1; no voters: 957e572b0e03411c9808169742a7aed6, a60bf7f6826245bf8fcd750e64815f22
I20260806 18:33:29.987735 32133 raft_consensus.cc:2726] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:33:30.036093 32134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:30.035625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:40839)
0806 18:33:30.035902 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:30.035969 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":82,"threads_started":1}
I20260806 18:33:30.243108 32135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:30.242667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:43669)
0806 18:33:30.242945 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:30.243003 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":78,"threads_started":1}
I20260806 18:33:30.890313 32136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:30.889860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:35409)
0806 18:33:30.890149 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:30.890208 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":91,"threads_started":1}
I20260806 18:33:31.037388 32137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:31.036859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:52067)
0806 18:33:31.037219 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:31.037282 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":79,"threads_started":1}
I20260806 18:33:31.244393 32138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:31.243859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:38451)
0806 18:33:31.244214 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:31.244286 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":92,"threads_started":1}
I20260806 18:33:31.875367 32139 raft_consensus.cc:482] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:33:31.875532 32139 raft_consensus.cc:504] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:33:31.875998 32139 leader_election.cc:288] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a60bf7f6826245bf8fcd750e64815f22 (127.7.67.195:45533), 957e572b0e03411c9808169742a7aed6 (127.7.67.194:46341)
W20260806 18:33:31.876772 31507 leader_election.cc:341] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 957e572b0e03411c9808169742a7aed6 (127.7.67.194:46341): Not found: Tablet not found: 105ee5f841f04251917ff6e853e860ee
W20260806 18:33:31.876891 31512 leader_election.cc:341] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a60bf7f6826245bf8fcd750e64815f22 (127.7.67.195:45533): Not found: Tablet not found: 105ee5f841f04251917ff6e853e860ee
I20260806 18:33:31.876988 31512 leader_election.cc:302] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 251302a9dffd4ce8a7c1330df743f0c1; no voters: 957e572b0e03411c9808169742a7aed6, a60bf7f6826245bf8fcd750e64815f22
I20260806 18:33:31.877151 32139 raft_consensus.cc:2726] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:33:31.891669 32140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:31.891181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:51619)
0806 18:33:31.891497 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:31.891561 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":79,"threads_started":1}
I20260806 18:33:32.038584 32141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:32.038127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:38271)
0806 18:33:32.038414 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:32.038475 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":97,"threads_started":1}
I20260806 18:33:32.245558 32142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:32.245122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:43115)
0806 18:33:32.245401 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:32.245458 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":93,"threads_started":1}
I20260806 18:33:32.892881 32143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:32.892416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:48761)
0806 18:33:32.892713 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:32.892775 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":91,"threads_started":1}
I20260806 18:33:33.039861 32146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:33.039339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:42079)
0806 18:33:33.039674 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:33.039738 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":111,"threads_started":1}
I20260806 18:33:33.246922 32147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:33.246374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:42401)
0806 18:33:33.246694 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:33.246759 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":104,"threads_started":1}
I20260806 18:33:33.894222 32148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:33.893745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:43819)
0806 18:33:33.894049 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:33.894114 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":84,"threads_started":1}
I20260806 18:33:34.041142 32149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:34.040680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:56645)
0806 18:33:34.040971 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:34.041036 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":82,"threads_started":1}
I20260806 18:33:34.248136 32150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:34.247678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:44159)
0806 18:33:34.247967 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:34.248034 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":91,"threads_started":1}
I20260806 18:33:34.895674 32151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:34.895149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:38711)
0806 18:33:34.895494 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:34.895564 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":83,"threads_started":1}
I20260806 18:33:35.042402 32152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:35.041960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:49039)
0806 18:33:35.042231 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:35.042293 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":76,"threads_started":1}
I20260806 18:33:35.249436 32154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:35.248973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:41937)
0806 18:33:35.249268 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:35.249330 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":101,"threads_started":1}
I20260806 18:33:35.896858 32156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:35.896439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:50815)
0806 18:33:35.896690 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:35.896746 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":76,"threads_started":1}
I20260806 18:33:36.043710 32157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:36.043214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:49197)
0806 18:33:36.043541 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:36.043604 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":88,"threads_started":1}
I20260806 18:33:36.250763 32159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:36.250307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:47815)
0806 18:33:36.250584 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:36.250654 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":81,"threads_started":1}
I20260806 18:33:36.898134 32160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:36.897639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:36179)
0806 18:33:36.897978 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:36.898034 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":79,"threads_started":1}
I20260806 18:33:37.044981 32161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:37.044498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:48779)
0806 18:33:37.044790 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:37.044866 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":81,"threads_started":1}
I20260806 18:33:37.252055 32163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:37.251599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:39703)
0806 18:33:37.251888 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:37.251949 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":92,"threads_started":1}
I20260806 18:33:37.603410 32165 raft_consensus.cc:482] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:33:37.603551 32165 raft_consensus.cc:504] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60bf7f6826245bf8fcd750e64815f22" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 45533 } } peers { permanent_uuid: "957e572b0e03411c9808169742a7aed6" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46341 } } peers { permanent_uuid: "251302a9dffd4ce8a7c1330df743f0c1" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 36471 } }
I20260806 18:33:37.603991 32165 leader_election.cc:288] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a60bf7f6826245bf8fcd750e64815f22 (127.7.67.195:45533), 957e572b0e03411c9808169742a7aed6 (127.7.67.194:46341)
W20260806 18:33:37.604781 31507 leader_election.cc:341] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 957e572b0e03411c9808169742a7aed6 (127.7.67.194:46341): Not found: Tablet not found: 105ee5f841f04251917ff6e853e860ee
W20260806 18:33:37.604914 31512 leader_election.cc:341] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a60bf7f6826245bf8fcd750e64815f22 (127.7.67.195:45533): Not found: Tablet not found: 105ee5f841f04251917ff6e853e860ee
I20260806 18:33:37.605005 31512 leader_election.cc:302] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 251302a9dffd4ce8a7c1330df743f0c1; no voters: 957e572b0e03411c9808169742a7aed6, a60bf7f6826245bf8fcd750e64815f22
I20260806 18:33:37.605162 32165 raft_consensus.cc:2726] T 105ee5f841f04251917ff6e853e860ee P 251302a9dffd4ce8a7c1330df743f0c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:33:37.899523 32168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:37.899034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:56641)
0806 18:33:37.899333 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:37.899405 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":81,"threads_started":1}
I20260806 18:33:38.046286 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:38.045803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:46121)
0806 18:33:38.046116 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:38.046179 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":107,"threads_started":1}
I20260806 18:33:38.253268 32171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:38.252820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:55963)
0806 18:33:38.253098 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:38.253160 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":91,"threads_started":1}
I20260806 18:33:38.900884 32173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:38.900473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.193:51951)
0806 18:33:38.900729 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:38.900784 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":74,"threads_started":1}
I20260806 18:33:39.047717 32174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:39.047257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.194:45611)
0806 18:33:39.047537 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:39.047598 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":100,"threads_started":1}
I20260806 18:33:39.245642 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 31415
I20260806 18:33:39.254765 32175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:39.254309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:44587 (local address 127.7.67.195:41705)
0806 18:33:39.254596 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:39.254660 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.254:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":93,"threads_started":1}
I20260806 18:33:39.261519 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 31677
I20260806 18:33:39.274264 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 31881
2026-08-06T18:33:39Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (60237 ms)
[----------] 30 tests from SecurityITest (247064 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:33:40 dist-test-slave-5bn2 krb5kdc[32178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:33:40 dist-test-slave-5bn2 krb5kdc[32178](info): set up 2 sockets
Aug 06 18:33:40 dist-test-slave-5bn2 krb5kdc[32178](info): commencing operation
krb5kdc: starting...
W20260806 18:33:41.326263 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:33:41 dist-test-slave-5bn2 krb5kdc[32178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:33:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:33:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:33:41.449584 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34821
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:38321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:34821
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:33:41.566481 32198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:41.566763 32198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:41.566823 32198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:41.570819 32198 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:33:41.570926 32198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:41.570950 32198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:41.570986 32198 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:33:41.571019 32198 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:33:41.576418 32198 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:38321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:34821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:41.577499 32198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:41.578570 32198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:33:41.587678 32198 server_base.cc:1034] running on GCE node
W20260806 18:33:41.587618 32206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:41.587644 32211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:41.587625 32209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:41.588480 32198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:41.589600 32198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:41.590792 32198 hybrid_clock.cc:648] HybridClock initialized: now 1786041221590767 us; error 39 us; skew 500 ppm
Aug 06 18:33:41 dist-test-slave-5bn2 krb5kdc[32178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041221, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:41.593793 32198 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:33:41.594704 32198 webserver.cc:460] Webserver started at http://127.7.67.254:38229/ using document root <none> and password file <none>
I20260806 18:33:41.595291 32198 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:41.595362 32198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:41.595621 32198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:41.597694 32198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "e894472b045740548bf69ed62610b6b6"
format_stamp: "Formatted at 2026-08-06 18:33:41 on dist-test-slave-5bn2"
server_key: "2ecf8c11d599a4ef566cab1026b36e1e"
server_key_iv: "d64927dfb10b1d5326b544957a758a55"
server_key_version: "encryptionkey@0"
I20260806 18:33:41.598250 32198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "e894472b045740548bf69ed62610b6b6"
format_stamp: "Formatted at 2026-08-06 18:33:41 on dist-test-slave-5bn2"
server_key: "2ecf8c11d599a4ef566cab1026b36e1e"
server_key_iv: "d64927dfb10b1d5326b544957a758a55"
server_key_version: "encryptionkey@0"
I20260806 18:33:41.601938 32198 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:33:41.604511 32220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:41.605736 32198 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:33:41.605859 32198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "e894472b045740548bf69ed62610b6b6"
format_stamp: "Formatted at 2026-08-06 18:33:41 on dist-test-slave-5bn2"
server_key: "2ecf8c11d599a4ef566cab1026b36e1e"
server_key_iv: "d64927dfb10b1d5326b544957a758a55"
server_key_version: "encryptionkey@0"
I20260806 18:33:41.605957 32198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:41.617897 32198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:41.620961 32198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:41.621116 32198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:41.627926 32198 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:34821
I20260806 18:33:41.628001 32290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:34821 every 8 connection(s)
I20260806 18:33:41.628911 32198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:33:41.631879 32295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:41.636425 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 32198
I20260806 18:33:41.636564 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:33:41.636876 7439 external_mini_cluster.cc:1428] Setting key 04e5a63bffb38ec57c46813a0c994434
I20260806 18:33:41.637948 32295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6: Bootstrap starting.
I20260806 18:33:41.640336 32295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:41.641227 32295 log.cc:824] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:41.643368 32295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6: No bootstrap required, opened a new log
Aug 06 18:33:41 dist-test-slave-5bn2 krb5kdc[32178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:33:41.647886 32295 raft_consensus.cc:348] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e894472b045740548bf69ed62610b6b6" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34821 } }
I20260806 18:33:41.648185 32295 raft_consensus.cc:374] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:41.648247 32295 raft_consensus.cc:729] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e894472b045740548bf69ed62610b6b6, State: Initialized, Role: FOLLOWER
I20260806 18:33:41.648765 32295 consensus_queue.cc:260] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e894472b045740548bf69ed62610b6b6" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34821 } }
I20260806 18:33:41.648924 32295 raft_consensus.cc:388] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:33:41.648994 32295 raft_consensus.cc:482] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:33:41.649119 32295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:41.650341 32295 raft_consensus.cc:504] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e894472b045740548bf69ed62610b6b6" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34821 } }
I20260806 18:33:41.650722 32295 leader_election.cc:302] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e894472b045740548bf69ed62610b6b6; no voters:
I20260806 18:33:41.651118 32295 leader_election.cc:288] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:33:41.651218 32304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:33:41.651465 32304 raft_consensus.cc:686] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6 [term 1 LEADER]: Becoming Leader. State: Replica: e894472b045740548bf69ed62610b6b6, State: Running, Role: LEADER
I20260806 18:33:41.651805 32304 consensus_queue.cc:237] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e894472b045740548bf69ed62610b6b6" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34821 } }
I20260806 18:33:41.651922 32301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:41.638329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40858 (local address 127.7.67.254:34821)
0806 18:33:41.638742 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:41.638757 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:41.638793 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:33:41.639462 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:41.639480 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:41.639801 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:41.640074 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:41.641021 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:41.642083 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:41.643255 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:41.643778 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:41.643909 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:41.646609 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:41.646616 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:41.646634 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:41.646668 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:41.648592 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:41.649049 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:41.649055 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:41.649062 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:41.649214 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:41.649485 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:41.649489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:41.649491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:41.650039 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:41.650272 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:41.650503 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 18:33:41.650815 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":120,"threads_started":1}
I20260806 18:33:41.652472 32295 sys_catalog.cc:564] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:33:41.653252 32309 sys_catalog.cc:455] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e894472b045740548bf69ed62610b6b6. Latest consensus state: current_term: 1 leader_uuid: "e894472b045740548bf69ed62610b6b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e894472b045740548bf69ed62610b6b6" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34821 } } }
I20260806 18:33:41.653385 32309 sys_catalog.cc:458] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:41.654157 32308 sys_catalog.cc:455] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e894472b045740548bf69ed62610b6b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e894472b045740548bf69ed62610b6b6" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34821 } } }
I20260806 18:33:41.654312 32308 sys_catalog.cc:458] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:41.654983 32322 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:33:41.658191 32322 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:33:41.664016 32322 catalog_manager.cc:1347] Generated new cluster ID: b76d6c9c3b5f4c3eacf514f5ee46a87f
I20260806 18:33:41.664079 32322 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:33:41.674903 32322 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:33:41.675874 32322 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:33:41.684880 32322 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6: Generated new TSK 0
I20260806 18:33:41.685597 32322 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
I20260806 18:33:41.717445 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:34821
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:38321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:33:41.830049 32343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:41.830324 32343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:41.830364 32343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:41.834275 32343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:41.834328 32343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:41.834388 32343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:33:41.839535 32343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:38321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:34821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:41.840546 32343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:41.841542 32343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:41.851034 32354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:41.851162 32343 server_base.cc:1034] running on GCE node
W20260806 18:33:41.851002 32351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:41.851022 32349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:41.851604 32343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:41.852240 32343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:41.853410 32343 hybrid_clock.cc:648] HybridClock initialized: now 1786041221853390 us; error 30 us; skew 500 ppm
Aug 06 18:33:41 dist-test-slave-5bn2 krb5kdc[32178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041221, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:41.856148 32343 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:33:41.856948 32343 webserver.cc:460] Webserver started at http://127.7.67.193:33353/ using document root <none> and password file <none>
I20260806 18:33:41.857465 32343 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:41.857523 32343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:41.857692 32343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:41.859733 32343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "71907a1eda3d45cab296fdb880f2e2a0"
format_stamp: "Formatted at 2026-08-06 18:33:41 on dist-test-slave-5bn2"
server_key: "cad9d07b886603f73dc02b9f712e2f47"
server_key_iv: "8a5cf7f5f681e9069f6b09a77950ae9d"
server_key_version: "encryptionkey@0"
I20260806 18:33:41.860252 32343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "71907a1eda3d45cab296fdb880f2e2a0"
format_stamp: "Formatted at 2026-08-06 18:33:41 on dist-test-slave-5bn2"
server_key: "cad9d07b886603f73dc02b9f712e2f47"
server_key_iv: "8a5cf7f5f681e9069f6b09a77950ae9d"
server_key_version: "encryptionkey@0"
I20260806 18:33:41.863862 32343 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:33:41.866267 32364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:41.867491 32343 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:33:41.867599 32343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "71907a1eda3d45cab296fdb880f2e2a0"
format_stamp: "Formatted at 2026-08-06 18:33:41 on dist-test-slave-5bn2"
server_key: "cad9d07b886603f73dc02b9f712e2f47"
server_key_iv: "8a5cf7f5f681e9069f6b09a77950ae9d"
server_key_version: "encryptionkey@0"
I20260806 18:33:41.867692 32343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:41.898180 32343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:41.901262 32343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:41.901450 32343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:41.902081 32343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:41.902952 32343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:41.903000 32343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:41.903067 32343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:41.903091 32343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:41.911641 32343 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:39643
I20260806 18:33:41.911667 32544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:39643 every 8 connection(s)
I20260806 18:33:41.912513 32343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:33:41.914803 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 32343
I20260806 18:33:41.914945 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:33:41.915256 7439 external_mini_cluster.cc:1428] Setting key e0f3fa51a24c29dd17ea01b55b04056d
Aug 06 18:33:41 dist-test-slave-5bn2 krb5kdc[32178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041221, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.194@KRBTEST.COM; defaulting to no policy
I20260806 18:33:41.925307 32301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:41.914244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:52531 (local address 127.7.67.254:34821)
0806 18:33:41.914391 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:41.914396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:41.915349 (+ 953us) server_negotiation.cc:407] Connection header received
0806 18:33:41.916460 (+ 1111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:41.916464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:41.916516 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:41.916588 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:41.917734 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:41.918248 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:41.919405 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:41.919634 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:41.919721 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:41.922020 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:41.922023 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:41.922025 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:41.922055 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:41.923357 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:41.923827 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:41.923830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:41.923832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:41.923912 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:41.924231 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:41.924235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:41.924236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:41.924415 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:41.924501 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:41.924996 (+ 495us) server_negotiation.cc:299] Negotiation successful
0806 18:33:41.925137 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:33:41.926105 32548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:41.914639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34821 (local address 127.7.67.193:52531)
0806 18:33:41.915175 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:41.915237 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:41.916261 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:41.916772 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:41.916779 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:41.917125 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:41.917592 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:41.917606 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:41.918367 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:41.918371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:41.919291 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:41.919301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:41.919772 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:41.919776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:41.919930 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:41.920470 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:33:41.920486 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:41.921865 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:41.923462 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:41.923467 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:41.923482 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:41.923734 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:41.923994 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:41.923997 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:41.923999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:41.924143 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:41.924517 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:41.924524 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:41.924767 (+ 243us) client_negotiation.cc:769] Sending connection context
0806 18:33:41.924935 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 18:33:41.925254 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":141,"threads_started":1}
I20260806 18:33:41.927275 32545 heartbeater.cc:347] Connected to a master server at 127.7.67.254:34821
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.194@KRBTEST.COM" created.
I20260806 18:33:41.927557 32545 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:41.928066 32545 heartbeater.cc:511] Master 127.7.67.254:34821 requested a full tablet report, sending...
I20260806 18:33:41.929565 32244 ts_manager.cc:194] Registered new tserver with Master: 71907a1eda3d45cab296fdb880f2e2a0 (127.7.67.193:39643)
I20260806 18:33:41.930999 32244 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:52531
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
I20260806 18:33:41.943149 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:34821
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:38321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:33:42.056667 32552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:42.056958 32552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:42.057005 32552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:42.061172 32552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:42.061244 32552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:42.061303 32552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:33:42.066705 32552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:38321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:34821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:42.067854 32552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:42.068872 32552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:42.079036 32567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:42.079015 32563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:42.079025 32552 server_base.cc:1034] running on GCE node
W20260806 18:33:42.079006 32564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:42.079638 32552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:42.080312 32552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:42.081485 32552 hybrid_clock.cc:648] HybridClock initialized: now 1786041222081463 us; error 32 us; skew 500 ppm
Aug 06 18:33:42 dist-test-slave-5bn2 krb5kdc[32178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041222, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:42.084316 32552 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:33:42.085249 32552 webserver.cc:460] Webserver started at http://127.7.67.194:46667/ using document root <none> and password file <none>
I20260806 18:33:42.085812 32552 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:42.085883 32552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:42.086110 32552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:42.088304 32552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "283f1820d35142698bf11bc21559334d"
format_stamp: "Formatted at 2026-08-06 18:33:42 on dist-test-slave-5bn2"
server_key: "e9c55b8b74ba5f0d5261f437c65a5e2b"
server_key_iv: "dc05718477ee6e43052ad25df7bf324d"
server_key_version: "encryptionkey@0"
I20260806 18:33:42.088871 32552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "283f1820d35142698bf11bc21559334d"
format_stamp: "Formatted at 2026-08-06 18:33:42 on dist-test-slave-5bn2"
server_key: "e9c55b8b74ba5f0d5261f437c65a5e2b"
server_key_iv: "dc05718477ee6e43052ad25df7bf324d"
server_key_version: "encryptionkey@0"
I20260806 18:33:42.092700 32552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:33:42.095160 32576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:42.096354 32552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:33:42.096470 32552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "283f1820d35142698bf11bc21559334d"
format_stamp: "Formatted at 2026-08-06 18:33:42 on dist-test-slave-5bn2"
server_key: "e9c55b8b74ba5f0d5261f437c65a5e2b"
server_key_iv: "dc05718477ee6e43052ad25df7bf324d"
server_key_version: "encryptionkey@0"
I20260806 18:33:42.096571 32552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:42.122362 32552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:42.125680 32552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:42.125880 32552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:42.126538 32552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:42.127458 32552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:42.127519 32552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:42.127578 32552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:42.127602 32552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:42.136394 32552 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:39231
I20260806 18:33:42.136415 32751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:39231 every 8 connection(s)
I20260806 18:33:42.137373 32552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:33:42.140027 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 32552
I20260806 18:33:42.140177 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:33:42.140522 7439 external_mini_cluster.cc:1428] Setting key c3ef71a15e907527784bde1dec707401
Aug 06 18:33:42 dist-test-slave-5bn2 krb5kdc[32178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041222, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:33:42.151880 32301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:42.139189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:48045 (local address 127.7.67.254:34821)
0806 18:33:42.139316 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:42.139320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:42.140272 (+ 952us) server_negotiation.cc:407] Connection header received
0806 18:33:42.141382 (+ 1110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:42.141386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:42.141443 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:42.141518 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:42.142955 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.143479 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.144843 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.145156 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.145261 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:42.148051 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:42.148056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:42.148058 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:42.148087 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:42.149651 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:42.150172 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:42.150176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:42.150179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:42.150271 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:42.150588 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:42.150592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:42.150593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:42.150836 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:42.150965 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:42.151565 (+ 600us) server_negotiation.cc:299] Negotiation successful
0806 18:33:42.151707 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
WARNING: no policy specified for kudu/127.7.67.195@KRBTEST.COM; defaulting to no policy
I20260806 18:33:42.152666 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:42.139565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34821 (local address 127.7.67.194:48045)
0806 18:33:42.140094 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:42.140160 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:42.141175 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:42.141699 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:42.141707 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:42.142133 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:42.142795 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:42.142815 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.143604 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.143607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:42.144718 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:42.144729 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.145286 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.145289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:42.145431 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:42.145997 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:33:42.146013 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:42.147855 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:42.149775 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:42.149780 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:42.149794 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:42.150060 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:42.150351 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:42.150353 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:42.150354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:42.150505 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:42.151002 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:42.151010 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:42.151295 (+ 285us) client_negotiation.cc:769] Sending connection context
0806 18:33:42.151466 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 18:33:42.151758 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":109,"threads_started":1}
I20260806 18:33:42.154027 32757 heartbeater.cc:347] Connected to a master server at 127.7.67.254:34821
I20260806 18:33:42.154273 32757 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.195@KRBTEST.COM" created.
I20260806 18:33:42.154945 32757 heartbeater.cc:511] Master 127.7.67.254:34821 requested a full tablet report, sending...
I20260806 18:33:42.156057 32244 ts_manager.cc:194] Registered new tserver with Master: 283f1820d35142698bf11bc21559334d (127.7.67.194:39231)
I20260806 18:33:42.156644 32244 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:48045
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
I20260806 18:33:42.169950 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:34821
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:38321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:33:42.285473 32762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:42.285776 32762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:42.285820 32762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:42.289927 32762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:42.290001 32762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:42.290082 32762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:33:42.295572 32762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:38321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:34821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:42.296866 32762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:42.297927 32762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:42.307883 305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:42.307945 300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:42.307883 301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:42.308279 32762 server_base.cc:1034] running on GCE node
I20260806 18:33:42.308673 32762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:42.309278 32762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:42.310454 32762 hybrid_clock.cc:648] HybridClock initialized: now 1786041222310434 us; error 30 us; skew 500 ppm
Aug 06 18:33:42 dist-test-slave-5bn2 krb5kdc[32178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041222, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:42.313253 32762 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:33:42.314128 32762 webserver.cc:460] Webserver started at http://127.7.67.195:41947/ using document root <none> and password file <none>
I20260806 18:33:42.314695 32762 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:42.314759 32762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:42.314996 32762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:42.317167 32762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "db8318814ec643098fbaa7ba24632390"
format_stamp: "Formatted at 2026-08-06 18:33:42 on dist-test-slave-5bn2"
server_key: "20e1aca58440d1371b4931200f0cd06e"
server_key_iv: "5572666add32a0384686e88b088ed178"
server_key_version: "encryptionkey@0"
I20260806 18:33:42.317713 32762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "db8318814ec643098fbaa7ba24632390"
format_stamp: "Formatted at 2026-08-06 18:33:42 on dist-test-slave-5bn2"
server_key: "20e1aca58440d1371b4931200f0cd06e"
server_key_iv: "5572666add32a0384686e88b088ed178"
server_key_version: "encryptionkey@0"
I20260806 18:33:42.321524 32762 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 18:33:42.324024 314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:42.325248 32762 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:33:42.325354 32762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "db8318814ec643098fbaa7ba24632390"
format_stamp: "Formatted at 2026-08-06 18:33:42 on dist-test-slave-5bn2"
server_key: "20e1aca58440d1371b4931200f0cd06e"
server_key_iv: "5572666add32a0384686e88b088ed178"
server_key_version: "encryptionkey@0"
I20260806 18:33:42.325448 32762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:42.344039 32762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:42.347410 32762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:42.347582 32762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:42.348186 32762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:42.349042 32762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:42.349092 32762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:42.349134 32762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:42.349155 32762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:42.358222 32762 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:36263
I20260806 18:33:42.358264 475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:36263 every 8 connection(s)
I20260806 18:33:42.359174 32762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:33:42.367516 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 32762
I20260806 18:33:42.367671 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:33:42.367972 7439 external_mini_cluster.cc:1428] Setting key 0acb868fae6afb1d31631b0a2526fa44
Aug 06 18:33:42 dist-test-slave-5bn2 krb5kdc[32178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041222, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:33:42.373153 32301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:42.361021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:48017 (local address 127.7.67.254:34821)
0806 18:33:42.361160 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:42.361164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:42.362042 (+ 878us) server_negotiation.cc:407] Connection header received
0806 18:33:42.363458 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:42.363461 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:42.363524 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:42.363608 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:42.364828 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.365369 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.366514 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.366737 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.366827 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:42.369642 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:42.369645 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:42.369647 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:42.369672 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:42.371134 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:42.371598 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:42.371600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:42.371602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:42.371680 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:42.372001 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:42.372004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:42.372006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:42.372259 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:42.372380 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:42.372893 (+ 513us) server_negotiation.cc:299] Negotiation successful
0806 18:33:42.373025 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:33:42.373978 479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:42.361422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34821 (local address 127.7.67.195:48017)
0806 18:33:42.361881 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:42.361926 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:42.363238 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:42.363798 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:42.363806 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:42.364172 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:42.364678 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:42.364693 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.365492 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.365494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:42.366387 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:42.366397 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.366910 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.366915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:42.367081 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:42.367786 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:33:42.367804 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:42.369495 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:42.371237 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:42.371242 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:42.371255 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:42.371518 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:42.371745 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:42.371747 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:42.371749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:42.371878 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:42.372374 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:42.372379 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:42.372635 (+ 256us) client_negotiation.cc:769] Sending connection context
0806 18:33:42.372812 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 18:33:42.373117 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":124,"threads_started":1}
I20260806 18:33:42.375103 476 heartbeater.cc:347] Connected to a master server at 127.7.67.254:34821
I20260806 18:33:42.375339 476 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:42.375851 476 heartbeater.cc:511] Master 127.7.67.254:34821 requested a full tablet report, sending...
I20260806 18:33:42.376904 32244 ts_manager.cc:194] Registered new tserver with Master: db8318814ec643098fbaa7ba24632390 (127.7.67.195:36263)
I20260806 18:33:42.377470 32244 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:48017
I20260806 18:33:42.381839 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:33:42.390997 32301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:42.383824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40870 (local address 127.7.67.254:34821)
0806 18:33:42.383936 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:42.383941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:42.384161 (+ 220us) server_negotiation.cc:407] Connection header received
0806 18:33:42.384278 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:42.384281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:42.384321 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:42.384385 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:42.385178 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.385685 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.386524 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.386737 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.386824 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:42.388153 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:42.388156 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:42.388158 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:42.388181 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:42.389378 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:42.389772 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:42.389775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:42.389777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:42.389855 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:42.390116 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:42.390120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:42.390121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:42.390347 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:42.390469 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:42.390697 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 18:33:42.390810 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:33:42.393883 32244 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:33:42.396936 32244 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:33:42.411202 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:42.405312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:39231 (local address 127.0.0.1:40450)
0806 18:33:42.405663 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:42.405718 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:42.406116 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:42.406886 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:42.406893 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:42.406917 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:42.407022 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:42.407027 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.408151 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.408154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:42.409348 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:42.409359 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.410674 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.410676 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:42.410755 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:42.410784 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:33:42.410924 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 18:33:42.411052 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":88,"threads_started":1}
I20260806 18:33:42.411213 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:42.405487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40450 (local address 127.7.67.194:39231)
0806 18:33:42.405926 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:42.405939 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:42.406143 (+ 204us) server_negotiation.cc:407] Connection header received
0806 18:33:42.406318 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:42.406325 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:42.406547 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:42.406700 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:42.407143 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.408035 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.409473 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.410337 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.410454 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:42.410615 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:42.410967 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 18:33:42.411064 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":125,"threads_started":1}
I20260806 18:33:42.411477 496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:42.405653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:36263 (local address 127.0.0.1:43396)
0806 18:33:42.406293 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:42.406306 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:42.406392 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:42.407435 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:42.407438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:42.407467 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:42.407633 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:42.407639 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.409143 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.409146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:42.410097 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:42.410108 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.411100 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.411103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:42.411190 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:42.411216 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:33:42.411265 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 18:33:42.411333 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"mutex_wait_us":66,"spinlock_wait_cycles":896,"thread_start_us":101,"threads_started":1}
I20260806 18:33:42.411954 500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:42.406039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:39643 (local address 127.0.0.1:52014)
0806 18:33:42.406640 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:42.406653 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:42.406724 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:42.407413 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:42.407416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:42.407435 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:42.407619 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:42.407626 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.409102 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.409106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:42.410479 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:42.410490 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.411550 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.411554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:42.411645 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:42.411668 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:33:42.411747 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 18:33:42.411819 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":138,"spinlock_wait_cycles":4864,"thread_start_us":110,"threads_started":1}
I20260806 18:33:42.411993 503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:42.406504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43396 (local address 127.7.67.195:36263)
0806 18:33:42.407053 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:42.407064 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:42.407082 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:33:42.407145 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:42.407149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:42.407290 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:42.407762 (+ 472us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:42.407930 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.409014 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.410218 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.410952 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.411578 (+ 626us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:42.411696 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:42.411746 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 18:33:42.411833 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":149,"threads_started":1}
I20260806 18:33:42.413133 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:42.406564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52014 (local address 127.7.67.193:39643)
0806 18:33:42.406976 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:42.406991 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:42.407014 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:33:42.407088 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:42.407093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:42.407274 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:42.407437 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:42.407740 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.408940 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.410610 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.411429 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.411544 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:42.411684 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:42.411994 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 18:33:42.412995 (+ 1001us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":74,"threads_started":1}
I20260806 18:33:42.415176 32651 tablet_service.cc:1467] Processing CreateTablet for tablet 9082e0f846af414fab00c330319e510f (DEFAULT_TABLE table=test-table [id=33e3281a2546487eb46ab8c23200d636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:42.415179 383 tablet_service.cc:1467] Processing CreateTablet for tablet 9082e0f846af414fab00c330319e510f (DEFAULT_TABLE table=test-table [id=33e3281a2546487eb46ab8c23200d636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:42.415736 32437 tablet_service.cc:1467] Processing CreateTablet for tablet 9082e0f846af414fab00c330319e510f (DEFAULT_TABLE table=test-table [id=33e3281a2546487eb46ab8c23200d636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:42.416226 32651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9082e0f846af414fab00c330319e510f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:42.416266 383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9082e0f846af414fab00c330319e510f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:42.416687 32437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9082e0f846af414fab00c330319e510f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:42.422766 505 tablet_bootstrap.cc:492] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d: Bootstrap starting.
I20260806 18:33:42.424470 507 tablet_bootstrap.cc:492] T 9082e0f846af414fab00c330319e510f P 71907a1eda3d45cab296fdb880f2e2a0: Bootstrap starting.
I20260806 18:33:42.425629 505 tablet_bootstrap.cc:654] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:42.426568 505 log.cc:824] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:42.426633 507 tablet_bootstrap.cc:654] T 9082e0f846af414fab00c330319e510f P 71907a1eda3d45cab296fdb880f2e2a0: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:42.427222 508 tablet_bootstrap.cc:492] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390: Bootstrap starting.
I20260806 18:33:42.427392 507 log.cc:824] T 9082e0f846af414fab00c330319e510f P 71907a1eda3d45cab296fdb880f2e2a0: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:42.428719 505 tablet_bootstrap.cc:492] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d: No bootstrap required, opened a new log
I20260806 18:33:42.428925 505 ts_tablet_manager.cc:1397] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:33:42.429224 508 tablet_bootstrap.cc:654] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:42.429425 507 tablet_bootstrap.cc:492] T 9082e0f846af414fab00c330319e510f P 71907a1eda3d45cab296fdb880f2e2a0: No bootstrap required, opened a new log
I20260806 18:33:42.429610 507 ts_tablet_manager.cc:1397] T 9082e0f846af414fab00c330319e510f P 71907a1eda3d45cab296fdb880f2e2a0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260806 18:33:42.430840 508 log.cc:824] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:42.432520 508 tablet_bootstrap.cc:492] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390: No bootstrap required, opened a new log
I20260806 18:33:42.432708 508 ts_tablet_manager.cc:1397] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:33:42.432941 505 raft_consensus.cc:348] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283f1820d35142698bf11bc21559334d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39231 } } peers { permanent_uuid: "71907a1eda3d45cab296fdb880f2e2a0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39643 } } peers { permanent_uuid: "db8318814ec643098fbaa7ba24632390" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36263 } }
I20260806 18:33:42.433239 505 raft_consensus.cc:374] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:42.433300 505 raft_consensus.cc:729] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283f1820d35142698bf11bc21559334d, State: Initialized, Role: FOLLOWER
I20260806 18:33:42.433604 507 raft_consensus.cc:348] T 9082e0f846af414fab00c330319e510f P 71907a1eda3d45cab296fdb880f2e2a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283f1820d35142698bf11bc21559334d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39231 } } peers { permanent_uuid: "71907a1eda3d45cab296fdb880f2e2a0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39643 } } peers { permanent_uuid: "db8318814ec643098fbaa7ba24632390" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36263 } }
I20260806 18:33:42.433764 505 consensus_queue.cc:260] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283f1820d35142698bf11bc21559334d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39231 } } peers { permanent_uuid: "71907a1eda3d45cab296fdb880f2e2a0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39643 } } peers { permanent_uuid: "db8318814ec643098fbaa7ba24632390" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36263 } }
I20260806 18:33:42.433849 507 raft_consensus.cc:374] T 9082e0f846af414fab00c330319e510f P 71907a1eda3d45cab296fdb880f2e2a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:42.433895 507 raft_consensus.cc:729] T 9082e0f846af414fab00c330319e510f P 71907a1eda3d45cab296fdb880f2e2a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71907a1eda3d45cab296fdb880f2e2a0, State: Initialized, Role: FOLLOWER
I20260806 18:33:42.434365 507 consensus_queue.cc:260] T 9082e0f846af414fab00c330319e510f P 71907a1eda3d45cab296fdb880f2e2a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283f1820d35142698bf11bc21559334d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39231 } } peers { permanent_uuid: "71907a1eda3d45cab296fdb880f2e2a0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39643 } } peers { permanent_uuid: "db8318814ec643098fbaa7ba24632390" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36263 } }
I20260806 18:33:42.434480 32757 heartbeater.cc:503] Master 127.7.67.254:34821 was elected leader, sending a full tablet report...
I20260806 18:33:42.434830 505 ts_tablet_manager.cc:1428] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 18:33:42.435293 507 ts_tablet_manager.cc:1428] T 9082e0f846af414fab00c330319e510f P 71907a1eda3d45cab296fdb880f2e2a0: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260806 18:33:42.435453 32545 heartbeater.cc:503] Master 127.7.67.254:34821 was elected leader, sending a full tablet report...
I20260806 18:33:42.436235 508 raft_consensus.cc:348] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283f1820d35142698bf11bc21559334d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39231 } } peers { permanent_uuid: "71907a1eda3d45cab296fdb880f2e2a0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39643 } } peers { permanent_uuid: "db8318814ec643098fbaa7ba24632390" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36263 } }
I20260806 18:33:42.436513 508 raft_consensus.cc:374] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:42.436571 508 raft_consensus.cc:729] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db8318814ec643098fbaa7ba24632390, State: Initialized, Role: FOLLOWER
I20260806 18:33:42.436966 508 consensus_queue.cc:260] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283f1820d35142698bf11bc21559334d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39231 } } peers { permanent_uuid: "71907a1eda3d45cab296fdb880f2e2a0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39643 } } peers { permanent_uuid: "db8318814ec643098fbaa7ba24632390" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36263 } }
I20260806 18:33:42.437649 476 heartbeater.cc:503] Master 127.7.67.254:34821 was elected leader, sending a full tablet report...
I20260806 18:33:42.437925 508 ts_tablet_manager.cc:1428] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:33:42.447495 517 raft_consensus.cc:482] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:33:42.447674 517 raft_consensus.cc:504] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283f1820d35142698bf11bc21559334d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39231 } } peers { permanent_uuid: "71907a1eda3d45cab296fdb880f2e2a0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39643 } } peers { permanent_uuid: "db8318814ec643098fbaa7ba24632390" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36263 } }
I20260806 18:33:42.448770 517 leader_election.cc:288] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71907a1eda3d45cab296fdb880f2e2a0 (127.7.67.193:39643), db8318814ec643098fbaa7ba24632390 (127.7.67.195:36263)
I20260806 18:33:42.452587 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:42.449095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:51133 (local address 127.7.67.193:39643)
0806 18:33:42.449242 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:42.449247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:42.449264 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:33:42.449339 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:42.449342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:42.449394 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:42.449471 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:42.449758 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.450370 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.451562 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.452018 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.452112 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:42.452155 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:42.452359 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 18:33:42.452438 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:33:42.452589 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:42.448974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:39643 (local address 127.7.67.194:51133)
0806 18:33:42.449146 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:42.449160 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:42.449247 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:42.449476 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:42.449478 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:42.449491 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:42.449635 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:42.449642 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.450492 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.450495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:42.451442 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:42.451452 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.452131 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.452135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:42.452237 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:42.452263 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:33:42.452335 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 18:33:42.452428 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"spinlock_wait_cycles":768}
I20260806 18:33:42.452661 503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:42.449085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:44257 (local address 127.7.67.195:36263)
0806 18:33:42.449246 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:42.449250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:42.449521 (+ 271us) server_negotiation.cc:407] Connection header received
0806 18:33:42.449656 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:42.449660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:42.449710 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:42.449772 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:42.450052 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.450619 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.451706 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.452134 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.452248 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:42.452291 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:42.452489 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 18:33:42.452557 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 18:33:42.452924 521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:42.448998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:36263 (local address 127.7.67.194:44257)
0806 18:33:42.449434 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:42.449447 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:42.449562 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:42.449793 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:42.449796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:42.449814 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:42.449922 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:42.449929 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.450739 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.450742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:42.451586 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:42.451597 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.452247 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.452249 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:42.452334 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:42.452358 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:33:42.452439 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 18:33:42.452552 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"spinlock_wait_cycles":768,"thread_start_us":124,"threads_started":1}
I20260806 18:33:42.453295 32468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9082e0f846af414fab00c330319e510f" candidate_uuid: "283f1820d35142698bf11bc21559334d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71907a1eda3d45cab296fdb880f2e2a0" is_pre_election: true
I20260806 18:33:42.453532 419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9082e0f846af414fab00c330319e510f" candidate_uuid: "283f1820d35142698bf11bc21559334d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db8318814ec643098fbaa7ba24632390" is_pre_election: true
I20260806 18:33:42.453681 32468 raft_consensus.cc:2454] T 9082e0f846af414fab00c330319e510f P 71907a1eda3d45cab296fdb880f2e2a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 283f1820d35142698bf11bc21559334d in term 0.
I20260806 18:33:42.453819 419 raft_consensus.cc:2454] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 283f1820d35142698bf11bc21559334d in term 0.
I20260806 18:33:42.454241 32580 leader_election.cc:302] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 283f1820d35142698bf11bc21559334d, 71907a1eda3d45cab296fdb880f2e2a0; no voters:
I20260806 18:33:42.454509 517 raft_consensus.cc:2781] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:33:42.454600 517 raft_consensus.cc:482] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:33:42.454648 517 raft_consensus.cc:3037] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:42.455737 517 raft_consensus.cc:504] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283f1820d35142698bf11bc21559334d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39231 } } peers { permanent_uuid: "71907a1eda3d45cab296fdb880f2e2a0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39643 } } peers { permanent_uuid: "db8318814ec643098fbaa7ba24632390" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36263 } }
I20260806 18:33:42.456110 517 leader_election.cc:288] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [CANDIDATE]: Term 1 election: Requested vote from peers 71907a1eda3d45cab296fdb880f2e2a0 (127.7.67.193:39643), db8318814ec643098fbaa7ba24632390 (127.7.67.195:36263)
I20260806 18:33:42.456467 419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9082e0f846af414fab00c330319e510f" candidate_uuid: "283f1820d35142698bf11bc21559334d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db8318814ec643098fbaa7ba24632390"
I20260806 18:33:42.456594 419 raft_consensus.cc:3037] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:42.456650 32468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9082e0f846af414fab00c330319e510f" candidate_uuid: "283f1820d35142698bf11bc21559334d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71907a1eda3d45cab296fdb880f2e2a0"
I20260806 18:33:42.456771 32468 raft_consensus.cc:3037] T 9082e0f846af414fab00c330319e510f P 71907a1eda3d45cab296fdb880f2e2a0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:42.457919 419 raft_consensus.cc:2454] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283f1820d35142698bf11bc21559334d in term 1.
I20260806 18:33:42.457993 32468 raft_consensus.cc:2454] T 9082e0f846af414fab00c330319e510f P 71907a1eda3d45cab296fdb880f2e2a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283f1820d35142698bf11bc21559334d in term 1.
I20260806 18:33:42.458258 32577 leader_election.cc:302] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 283f1820d35142698bf11bc21559334d, db8318814ec643098fbaa7ba24632390; no voters:
I20260806 18:33:42.458480 517 raft_consensus.cc:2781] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:33:42.458698 517 raft_consensus.cc:686] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [term 1 LEADER]: Becoming Leader. State: Replica: 283f1820d35142698bf11bc21559334d, State: Running, Role: LEADER
I20260806 18:33:42.459069 517 consensus_queue.cc:237] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283f1820d35142698bf11bc21559334d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39231 } } peers { permanent_uuid: "71907a1eda3d45cab296fdb880f2e2a0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39643 } } peers { permanent_uuid: "db8318814ec643098fbaa7ba24632390" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36263 } }
I20260806 18:33:42.462334 32241 catalog_manager.cc:5495] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d reported cstate change: term changed from 0 to 1, leader changed from <none> to 283f1820d35142698bf11bc21559334d (127.7.67.194). New cstate: current_term: 1 leader_uuid: "283f1820d35142698bf11bc21559334d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283f1820d35142698bf11bc21559334d" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 39231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71907a1eda3d45cab296fdb880f2e2a0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db8318814ec643098fbaa7ba24632390" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36263 } health_report { overall_health: UNKNOWN } } }
I20260806 18:33:42.477331 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:42.473893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40462 (local address 127.7.67.194:39231)
0806 18:33:42.473999 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:42.474003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:42.474021 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:33:42.474167 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:42.474169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:42.474225 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:42.474313 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:33:42.474515 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.475058 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.475998 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.476218 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.476296 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:42.476623 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:33:42.477042 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:33:42.477131 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:42.477167 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 18:33:42.477218 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 18:33:42.483327 419 raft_consensus.cc:1261] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390 [term 1 FOLLOWER]: Refusing update from remote peer 283f1820d35142698bf11bc21559334d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:33:42.483333 32468 raft_consensus.cc:1261] T 9082e0f846af414fab00c330319e510f P 71907a1eda3d45cab296fdb880f2e2a0 [term 1 FOLLOWER]: Refusing update from remote peer 283f1820d35142698bf11bc21559334d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:33:42.484038 522 consensus_queue.cc:1035] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71907a1eda3d45cab296fdb880f2e2a0" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:33:42.484225 517 consensus_queue.cc:1035] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db8318814ec643098fbaa7ba24632390" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:33:42.503108 32244 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:33:42.503320 32244 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:33:42.506016 32244 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e894472b045740548bf69ed62610b6b6: Sending DeleteTablet for 3 replicas of tablet 9082e0f846af414fab00c330319e510f
I20260806 18:33:42.506826 383 tablet_service.cc:1514] Processing DeleteTablet for tablet 9082e0f846af414fab00c330319e510f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:42 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:43396
I20260806 18:33:42.506937 32437 tablet_service.cc:1514] Processing DeleteTablet for tablet 9082e0f846af414fab00c330319e510f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:42 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:52014
I20260806 18:33:42.507393 542 tablet_replica.cc:330] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390: stopping tablet replica
I20260806 18:33:42.507443 546 tablet_replica.cc:330] T 9082e0f846af414fab00c330319e510f P 71907a1eda3d45cab296fdb880f2e2a0: stopping tablet replica
I20260806 18:33:42.507757 542 raft_consensus.cc:2229] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:33:42.507757 546 raft_consensus.cc:2229] T 9082e0f846af414fab00c330319e510f P 71907a1eda3d45cab296fdb880f2e2a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:33:42.507901 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 32343
I20260806 18:33:42.513203 542 raft_consensus.cc:2258] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:33:42.513172 32651 tablet_service.cc:1514] Processing DeleteTablet for tablet 9082e0f846af414fab00c330319e510f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:42 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:40450
I20260806 18:33:42.515667 542 ts_tablet_manager.cc:1905] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:33:42.517938 542 ts_tablet_manager.cc:1918] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:33:42.518039 542 log.cc:1196] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/9082e0f846af414fab00c330319e510f
I20260806 18:33:42.518404 542 ts_tablet_manager.cc:1939] T 9082e0f846af414fab00c330319e510f P db8318814ec643098fbaa7ba24632390: Deleting consensus metadata
I20260806 18:33:42.518414 547 tablet_replica.cc:330] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d: stopping tablet replica
I20260806 18:33:42.518930 547 raft_consensus.cc:2229] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:33:42.519438 547 raft_consensus.cc:2258] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:33:42.519866 32221 catalog_manager.cc:4841] TS db8318814ec643098fbaa7ba24632390 (127.7.67.195:36263): tablet 9082e0f846af414fab00c330319e510f (table test-table [id=33e3281a2546487eb46ab8c23200d636]) successfully deleted
I20260806 18:33:42.520998 547 ts_tablet_manager.cc:1905] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 18:33:42.522383 32222 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.193:39643 (error 108)
I20260806 18:33:42.523501 547 ts_tablet_manager.cc:1918] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:33:42.523525 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:42.523194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:39643 (local address 127.0.0.1:52028)
0806 18:33:42.523346 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:42.523428 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:39643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 18:33:42.523557 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 32552
I20260806 18:33:42.523599 547 log.cc:1196] T 9082e0f846af414fab00c330319e510f P 283f1820d35142698bf11bc21559334d: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/9082e0f846af414fab00c330319e510f
W20260806 18:33:42.528878 32222 catalog_manager.cc:4584] TS 71907a1eda3d45cab296fdb880f2e2a0 (127.7.67.193:39643): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9082e0f846af414fab00c330319e510f: Network error: Client connection negotiation failed: client connection to 127.7.67.193:39643: connect: Connection refused (error 111)
I20260806 18:33:42.535017 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 32762
I20260806 18:33:42.542871 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:42.535133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:39231 (local address 127.0.0.1:40468)
0806 18:33:42.542636 (+ 7503us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:42.542728 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.194:39231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7367}
I20260806 18:33:42.550908 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 32198
2026-08-06T18:33:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2358 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T18:33:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:33:42Z Disabled control of system clock
I20260806 18:33:42.588694 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:35057
--webserver_interface=127.7.67.254
--webserver_port=0
--builtin_ntp_servers=127.7.67.212:45713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:35057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:33:42.704984 556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:42.705279 556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:42.705315 556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:42.709314 556 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:33:42.709383 556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:42.709398 556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:42.709412 556 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:33:42.709426 556 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:33:42.714217 556 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:45713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:35057
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:35057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:42.715410 556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:42.716795 556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:42.725127 563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:42.725114 564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:42.725114 566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:42.725363 556 server_base.cc:1034] running on GCE node
I20260806 18:33:42.726028 556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:42.727177 556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:42.728328 556 hybrid_clock.cc:648] HybridClock initialized: now 1786041222728311 us; error 34 us; skew 500 ppm
I20260806 18:33:42.730145 556 webserver.cc:460] Webserver started at http://127.7.67.254:45247/ using document root <none> and password file <none>
I20260806 18:33:42.730720 556 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:42.730784 556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:42.731024 556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:42.733125 556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "783a31bd1f934e5092b4cecb322d2ba2"
format_stamp: "Formatted at 2026-08-06 18:33:42 on dist-test-slave-5bn2"
server_key: "057b94feb896960dc6a54301d16fdddd"
server_key_iv: "caab1fef20bfc41a24b6d13186225750"
server_key_version: "encryptionkey@0"
I20260806 18:33:42.733709 556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "783a31bd1f934e5092b4cecb322d2ba2"
format_stamp: "Formatted at 2026-08-06 18:33:42 on dist-test-slave-5bn2"
server_key: "057b94feb896960dc6a54301d16fdddd"
server_key_iv: "caab1fef20bfc41a24b6d13186225750"
server_key_version: "encryptionkey@0"
I20260806 18:33:42.737455 556 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:33:42.739935 574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:42.741142 556 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:33:42.741257 556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "783a31bd1f934e5092b4cecb322d2ba2"
format_stamp: "Formatted at 2026-08-06 18:33:42 on dist-test-slave-5bn2"
server_key: "057b94feb896960dc6a54301d16fdddd"
server_key_iv: "caab1fef20bfc41a24b6d13186225750"
server_key_version: "encryptionkey@0"
I20260806 18:33:42.741367 556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:42.764338 556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:42.765129 556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:42.765327 556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:42.772519 556 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:35057
I20260806 18:33:42.772528 644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:35057 every 8 connection(s)
I20260806 18:33:42.773569 556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:33:42.775321 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 556
I20260806 18:33:42.775477 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:33:42.775813 7439 external_mini_cluster.cc:1428] Setting key 2f51bed492bcbc27ec8f692bfb45f7f7
I20260806 18:33:42.777441 646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:42.783408 646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2: Bootstrap starting.
I20260806 18:33:42.785247 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:42.777369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56576 (local address 127.7.67.254:35057)
0806 18:33:42.778073 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:42.778090 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:42.778129 (+ 39us) server_negotiation.cc:407] Connection header received
0806 18:33:42.778817 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:42.778834 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:42.779317 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:42.779649 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:42.780105 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.781072 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.781957 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:42.782452 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:42.782615 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:42.783519 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:42.783527 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:42.783543 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:42.783934 (+ 391us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:42.783989 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:42.784011 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:42.784097 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:42.784240 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 18:33:42.784499 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":145,"threads_started":1}
I20260806 18:33:42.785799 646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:42.786552 646 log.cc:824] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:42.788487 646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2: No bootstrap required, opened a new log
I20260806 18:33:42.792055 646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "783a31bd1f934e5092b4cecb322d2ba2" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 35057 } }
I20260806 18:33:42.792297 646 raft_consensus.cc:374] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:42.792356 646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 783a31bd1f934e5092b4cecb322d2ba2, State: Initialized, Role: FOLLOWER
I20260806 18:33:42.792765 646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "783a31bd1f934e5092b4cecb322d2ba2" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 35057 } }
I20260806 18:33:42.792909 646 raft_consensus.cc:388] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:33:42.792974 646 raft_consensus.cc:482] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:33:42.793076 646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:42.794032 646 raft_consensus.cc:504] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "783a31bd1f934e5092b4cecb322d2ba2" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 35057 } }
I20260806 18:33:42.794397 646 leader_election.cc:302] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 783a31bd1f934e5092b4cecb322d2ba2; no voters:
I20260806 18:33:42.794718 646 leader_election.cc:288] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:33:42.794809 651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:33:42.795065 651 raft_consensus.cc:686] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2 [term 1 LEADER]: Becoming Leader. State: Replica: 783a31bd1f934e5092b4cecb322d2ba2, State: Running, Role: LEADER
I20260806 18:33:42.795389 651 consensus_queue.cc:237] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "783a31bd1f934e5092b4cecb322d2ba2" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 35057 } }
I20260806 18:33:42.796033 646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:33:42.797109 654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 783a31bd1f934e5092b4cecb322d2ba2. Latest consensus state: current_term: 1 leader_uuid: "783a31bd1f934e5092b4cecb322d2ba2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "783a31bd1f934e5092b4cecb322d2ba2" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 35057 } } }
I20260806 18:33:42.797130 652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "783a31bd1f934e5092b4cecb322d2ba2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "783a31bd1f934e5092b4cecb322d2ba2" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 35057 } } }
I20260806 18:33:42.797257 652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:42.797259 654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:42.797883 665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:33:42.801177 665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:33:42.807375 665 catalog_manager.cc:1347] Generated new cluster ID: ac5e30fd91ab4019860f0f0c1a365f49
I20260806 18:33:42.807458 665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:33:42.829432 665 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:33:42.830448 665 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:33:42.835712 665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2: Generated new TSK 0
I20260806 18:33:42.836453 665 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:33:42.842058 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:35057
--builtin_ntp_servers=127.7.67.212:45713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:33:42.961043 678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:42.961321 678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:42.961361 678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:42.965502 678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:42.965575 678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:42.965631 678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:33:42.970571 678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:45713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:35057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:42.971748 678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:42.973215 678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:33:42.983736 678 server_base.cc:1034] running on GCE node
W20260806 18:33:42.983685 683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:42.983711 684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:42.983685 686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:42.984326 678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:42.984972 678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:42.986135 678 hybrid_clock.cc:648] HybridClock initialized: now 1786041222986115 us; error 33 us; skew 500 ppm
I20260806 18:33:42.987990 678 webserver.cc:460] Webserver started at http://127.7.67.193:41989/ using document root <none> and password file <none>
I20260806 18:33:42.988569 678 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:42.988638 678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:42.988849 678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:42.991050 678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "be2770f65a754de78cb74bfc99eac78e"
format_stamp: "Formatted at 2026-08-06 18:33:42 on dist-test-slave-5bn2"
server_key: "f236e6add16313eb8f7176c2f5f459f5"
server_key_iv: "78b5cf431c6c8c74c27844caf8856014"
server_key_version: "encryptionkey@0"
I20260806 18:33:42.991601 678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "be2770f65a754de78cb74bfc99eac78e"
format_stamp: "Formatted at 2026-08-06 18:33:42 on dist-test-slave-5bn2"
server_key: "f236e6add16313eb8f7176c2f5f459f5"
server_key_iv: "78b5cf431c6c8c74c27844caf8856014"
server_key_version: "encryptionkey@0"
I20260806 18:33:42.995340 678 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:33:42.997893 696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:42.999267 678 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:33:42.999398 678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "be2770f65a754de78cb74bfc99eac78e"
format_stamp: "Formatted at 2026-08-06 18:33:42 on dist-test-slave-5bn2"
server_key: "f236e6add16313eb8f7176c2f5f459f5"
server_key_iv: "78b5cf431c6c8c74c27844caf8856014"
server_key_version: "encryptionkey@0"
I20260806 18:33:42.999514 678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:43.021633 678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:43.022375 678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:43.022572 678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:43.023202 678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:43.024204 678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:43.024256 678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:43.024314 678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:43.024341 678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:43.032748 678 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:39549
I20260806 18:33:43.032783 871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:39549 every 8 connection(s)
I20260806 18:33:43.033670 678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:33:43.040220 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 678
I20260806 18:33:43.040355 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:33:43.040632 7439 external_mini_cluster.cc:1428] Setting key d81ccc87fb4939c1a55b5ce8dfde73df
I20260806 18:33:43.042670 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:35057
--builtin_ntp_servers=127.7.67.212:45713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:33:43.042892 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:43.035530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:54437 (local address 127.7.67.254:35057)
0806 18:33:43.035677 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:43.035682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:43.036800 (+ 1118us) server_negotiation.cc:407] Connection header received
0806 18:33:43.038176 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:43.038178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:43.038232 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:43.038308 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:43.039269 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.039793 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.040983 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.041214 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.041300 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:43.042382 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:43.042386 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:43.042389 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:43.042404 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:43.042431 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:43.042441 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:43.042516 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:43.042637 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 18:33:43.042707 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:33:43.043751 877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:43.035949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:35057 (local address 127.7.67.193:54437)
0806 18:33:43.036643 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:43.036685 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:43.037972 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:43.038474 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:43.038481 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:43.038568 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:43.039146 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:43.039163 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.039892 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.039894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:43.040855 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:43.040866 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.041369 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.041373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:43.041531 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:43.042118 (+ 587us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:33:43.042119 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:43.042161 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:33:43.042228 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:43.042515 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:43.042520 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:43.042523 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:33:43.042611 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 18:33:43.042912 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":126,"threads_started":1}
I20260806 18:33:43.044994 872 heartbeater.cc:347] Connected to a master server at 127.7.67.254:35057
I20260806 18:33:43.045235 872 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:43.045838 872 heartbeater.cc:511] Master 127.7.67.254:35057 requested a full tablet report, sending...
I20260806 18:33:43.047369 598 ts_manager.cc:194] Registered new tserver with Master: be2770f65a754de78cb74bfc99eac78e (127.7.67.193:39549)
I20260806 18:33:43.048527 598 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.193:54437
W20260806 18:33:43.160265 878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:43.160548 878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:43.160606 878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:43.164631 878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:43.164695 878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:43.164772 878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:33:43.169545 878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:45713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:35057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:43.170763 878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:43.172256 878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:43.182134 889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:43.182138 885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:43.182138 887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:43.182433 878 server_base.cc:1034] running on GCE node
I20260806 18:33:43.182886 878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:43.183554 878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:43.184727 878 hybrid_clock.cc:648] HybridClock initialized: now 1786041223184705 us; error 33 us; skew 500 ppm
I20260806 18:33:43.186590 878 webserver.cc:460] Webserver started at http://127.7.67.194:35255/ using document root <none> and password file <none>
I20260806 18:33:43.187217 878 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:43.187292 878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:43.187522 878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:43.189646 878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "d8426646565a4a49b183817a98b603a2"
format_stamp: "Formatted at 2026-08-06 18:33:43 on dist-test-slave-5bn2"
server_key: "2fc31ac1eea88c80109dccf0137ba8db"
server_key_iv: "787ae464898b77da4c284ea7f2427e7e"
server_key_version: "encryptionkey@0"
I20260806 18:33:43.190214 878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "d8426646565a4a49b183817a98b603a2"
format_stamp: "Formatted at 2026-08-06 18:33:43 on dist-test-slave-5bn2"
server_key: "2fc31ac1eea88c80109dccf0137ba8db"
server_key_iv: "787ae464898b77da4c284ea7f2427e7e"
server_key_version: "encryptionkey@0"
I20260806 18:33:43.193998 878 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:33:43.196633 899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:43.197922 878 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:33:43.198043 878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "d8426646565a4a49b183817a98b603a2"
format_stamp: "Formatted at 2026-08-06 18:33:43 on dist-test-slave-5bn2"
server_key: "2fc31ac1eea88c80109dccf0137ba8db"
server_key_iv: "787ae464898b77da4c284ea7f2427e7e"
server_key_version: "encryptionkey@0"
I20260806 18:33:43.198169 878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:43.214553 878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:43.215366 878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:43.215569 878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:43.216195 878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:43.217206 878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:43.217255 878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:43.217314 878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:43.217340 878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:43.226033 878 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:46031
I20260806 18:33:43.226070 1066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:46031 every 8 connection(s)
I20260806 18:33:43.227018 878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:33:43.231330 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 878
I20260806 18:33:43.231482 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:33:43.231809 7439 external_mini_cluster.cc:1428] Setting key 05e930ebc482a6aa3ab7e6da395182f1
I20260806 18:33:43.233958 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:35057
--builtin_ntp_servers=127.7.67.212:45713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:33:43.237280 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:43.228976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:58255 (local address 127.7.67.254:35057)
0806 18:33:43.229161 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:43.229166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:43.230277 (+ 1111us) server_negotiation.cc:407] Connection header received
0806 18:33:43.232011 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:43.232014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:43.232079 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:43.232155 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:43.233164 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.233731 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.235355 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.235584 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.235665 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:43.236852 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:43.236856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:43.236859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:43.236874 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:43.236901 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:43.236911 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:43.236978 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:43.237102 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 18:33:43.237164 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 18:33:43.238114 1071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:43.229395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:35057 (local address 127.7.67.194:58255)
0806 18:33:43.230127 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:43.230171 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:43.231726 (+ 1555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:43.232354 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:43.232361 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:43.232461 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:43.233026 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:43.233046 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.233887 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.233891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:43.235227 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:43.235240 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.235746 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.235750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:43.235962 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:43.236587 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:33:43.236589 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:43.236631 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:33:43.236698 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:43.236988 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:43.236993 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:43.236997 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:33:43.237083 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 18:33:43.237310 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":127,"threads_started":1}
I20260806 18:33:43.239295 1068 heartbeater.cc:347] Connected to a master server at 127.7.67.254:35057
I20260806 18:33:43.239540 1068 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:43.240083 1068 heartbeater.cc:511] Master 127.7.67.254:35057 requested a full tablet report, sending...
I20260806 18:33:43.241120 598 ts_manager.cc:194] Registered new tserver with Master: d8426646565a4a49b183817a98b603a2 (127.7.67.194:46031)
I20260806 18:33:43.241665 598 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.194:58255
W20260806 18:33:43.350883 1072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:43.351207 1072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:43.351274 1072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:43.355396 1072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:43.355480 1072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:43.355571 1072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:33:43.360419 1072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:45713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:35057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:43.361575 1072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:43.363070 1072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:33:43.372939 1072 server_base.cc:1034] running on GCE node
W20260806 18:33:43.372891 1081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:43.372923 1079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:43.372895 1085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:43.373575 1072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:43.374228 1072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:43.375386 1072 hybrid_clock.cc:648] HybridClock initialized: now 1786041223375368 us; error 32 us; skew 500 ppm
I20260806 18:33:43.377265 1072 webserver.cc:460] Webserver started at http://127.7.67.195:43301/ using document root <none> and password file <none>
I20260806 18:33:43.377882 1072 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:43.377959 1072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:43.378165 1072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:43.380313 1072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "010f090f5aad4992a456166d65056007"
format_stamp: "Formatted at 2026-08-06 18:33:43 on dist-test-slave-5bn2"
server_key: "69beb8c5f1585c81379e2e6737d75710"
server_key_iv: "f7f5499ab54dbe9acd4f7bfb5a415e2a"
server_key_version: "encryptionkey@0"
I20260806 18:33:43.380868 1072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "010f090f5aad4992a456166d65056007"
format_stamp: "Formatted at 2026-08-06 18:33:43 on dist-test-slave-5bn2"
server_key: "69beb8c5f1585c81379e2e6737d75710"
server_key_iv: "f7f5499ab54dbe9acd4f7bfb5a415e2a"
server_key_version: "encryptionkey@0"
I20260806 18:33:43.384585 1072 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:33:43.387055 1096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:43.388229 1072 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:33:43.388337 1072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "010f090f5aad4992a456166d65056007"
format_stamp: "Formatted at 2026-08-06 18:33:43 on dist-test-slave-5bn2"
server_key: "69beb8c5f1585c81379e2e6737d75710"
server_key_iv: "f7f5499ab54dbe9acd4f7bfb5a415e2a"
server_key_version: "encryptionkey@0"
I20260806 18:33:43.388448 1072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:43.416282 1072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:43.417061 1072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:43.417244 1072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:43.417862 1072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:43.418838 1072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:43.418920 1072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:43.418969 1072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:43.418998 1072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:43.427634 1072 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:43175
I20260806 18:33:43.427668 1251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:43175 every 8 connection(s)
I20260806 18:33:43.428555 1072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:33:43.432305 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 1072
I20260806 18:33:43.432415 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:33:43.432675 7439 external_mini_cluster.cc:1428] Setting key 439492efdb7276ab1db4044d1dfd7d3a
I20260806 18:33:43.437800 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:43.430450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:43525 (local address 127.7.67.254:35057)
0806 18:33:43.430623 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:43.430629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:43.431703 (+ 1074us) server_negotiation.cc:407] Connection header received
0806 18:33:43.433196 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:43.433199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:43.433249 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:43.433316 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:43.434336 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.435059 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.436150 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.436354 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.436422 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:43.437413 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:43.437417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:43.437420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:43.437433 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:43.437452 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:43.437460 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:43.437508 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:43.437631 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 18:33:43.437684 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 18:33:43.438671 1254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:43.430835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:35057 (local address 127.7.67.195:43525)
0806 18:33:43.431555 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:43.431599 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:43.432982 (+ 1383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:43.433502 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:43.433511 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:43.433624 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:43.434202 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:43.434219 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.435168 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.435171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:43.436057 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:43.436067 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.436460 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.436463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:43.436599 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:43.437158 (+ 559us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:33:43.437160 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:43.437202 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:33:43.437268 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:43.437520 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:43.437526 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:43.437531 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 18:33:43.437623 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 18:33:43.437868 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":116,"threads_started":1}
I20260806 18:33:43.439826 1252 heartbeater.cc:347] Connected to a master server at 127.7.67.254:35057
I20260806 18:33:43.440061 1252 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:43.440605 1252 heartbeater.cc:511] Master 127.7.67.254:35057 requested a full tablet report, sending...
I20260806 18:33:43.441596 598 ts_manager.cc:194] Registered new tserver with Master: 010f090f5aad4992a456166d65056007 (127.7.67.195:43175)
I20260806 18:33:43.442137 598 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.195:43525
I20260806 18:33:43.446547 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:33:43.451876 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:43.448475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56586 (local address 127.7.67.254:35057)
0806 18:33:43.448592 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:43.448595 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:43.448771 (+ 176us) server_negotiation.cc:407] Connection header received
0806 18:33:43.448868 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:43.448870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:43.448909 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:43.448972 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:43.449170 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.449674 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.450552 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.450768 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.450890 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:43.451563 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:43.451566 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:43.451568 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:43.451580 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:43.451599 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:43.451606 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:43.451663 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:43.451719 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 18:33:43.451764 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:33:43.454450 598 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:33:43.457438 598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:33:43.471732 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:43.465957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:43175 (local address 127.0.0.1:35428)
0806 18:33:43.466739 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:43.466752 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:43.466846 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:43.467507 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:43.467510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:43.467542 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:43.467808 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:43.467817 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.469308 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.469311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:43.470289 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:43.470299 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.471171 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.471175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:43.471283 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:43.471323 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 18:33:43.471436 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 18:33:43.471566 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":61,"spinlock_wait_cycles":2688,"thread_start_us":121,"threads_started":1}
I20260806 18:33:43.471750 1275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:43.466521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35428 (local address 127.7.67.195:43175)
0806 18:33:43.467049 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:43.467061 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:43.467087 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:33:43.467164 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:43.467169 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:43.467348 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:43.467659 (+ 311us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:43.467950 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.469152 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.470410 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.471052 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.471156 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:43.471348 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:43.471482 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 18:33:43.471584 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":131,"threads_started":1}
I20260806 18:33:43.472347 1267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:43.465705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:39549 (local address 127.0.0.1:43188)
0806 18:33:43.466208 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:43.466260 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:43.466398 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:43.467498 (+ 1100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:43.467504 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:43.467540 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:43.467808 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:43.467817 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.469950 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.469954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:43.470698 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:43.470708 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.471966 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.471970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:43.472057 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:43.472076 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:33:43.472141 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 18:33:43.472209 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":85,"spinlock_wait_cycles":11904,"thread_start_us":95,"threads_started":1}
I20260806 18:33:43.472401 1270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:43.466446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:46031 (local address 127.0.0.1:55796)
0806 18:33:43.467253 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:43.467266 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:43.467374 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:43.467809 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:43.467812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:43.467827 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:43.467925 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:43.467931 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.468871 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.468874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:43.470830 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:43.470834 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.472015 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.472019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:43.472107 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:43.472131 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:33:43.472200 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 18:33:43.472271 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"mutex_wait_us":896,"thread_start_us":93,"threads_started":1}
I20260806 18:33:43.472445 1268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:43.465914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43188 (local address 127.7.67.193:39549)
0806 18:33:43.466406 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:43.466419 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:43.466447 (+ 28us) server_negotiation.cc:407] Connection header received
0806 18:33:43.467087 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:43.467093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:43.467311 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:43.467462 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:43.467950 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.469152 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.470822 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.471613 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.471714 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:43.471899 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:43.472198 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 18:33:43.472293 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":161,"threads_started":1}
I20260806 18:33:43.472807 1276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:43.466724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55796 (local address 127.7.67.194:46031)
0806 18:33:43.467112 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:43.467123 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:43.467365 (+ 242us) server_negotiation.cc:407] Connection header received
0806 18:33:43.467488 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:43.467493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:43.467659 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:43.467794 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:43.468041 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.468765 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.470958 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.471662 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.471757 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:43.471941 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:43.472554 (+ 613us) server_negotiation.cc:299] Negotiation successful
0806 18:33:43.472647 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":135,"threads_started":1}
I20260806 18:33:43.475198 764 tablet_service.cc:1467] Processing CreateTablet for tablet c69ca4c8880f4293a5b8afe11199634f (DEFAULT_TABLE table=test-table [id=ae1f3bd434474e4cb573e799ce69c28d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:43.475255 1160 tablet_service.cc:1467] Processing CreateTablet for tablet c69ca4c8880f4293a5b8afe11199634f (DEFAULT_TABLE table=test-table [id=ae1f3bd434474e4cb573e799ce69c28d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:43.475256 970 tablet_service.cc:1467] Processing CreateTablet for tablet c69ca4c8880f4293a5b8afe11199634f (DEFAULT_TABLE table=test-table [id=ae1f3bd434474e4cb573e799ce69c28d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:43.476226 764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c69ca4c8880f4293a5b8afe11199634f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:43.476284 970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c69ca4c8880f4293a5b8afe11199634f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:43.476284 1160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c69ca4c8880f4293a5b8afe11199634f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:43.482961 1277 tablet_bootstrap.cc:492] T c69ca4c8880f4293a5b8afe11199634f P be2770f65a754de78cb74bfc99eac78e: Bootstrap starting.
I20260806 18:33:43.484237 1279 tablet_bootstrap.cc:492] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2: Bootstrap starting.
I20260806 18:33:43.484311 1278 tablet_bootstrap.cc:492] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007: Bootstrap starting.
I20260806 18:33:43.485280 1277 tablet_bootstrap.cc:654] T c69ca4c8880f4293a5b8afe11199634f P be2770f65a754de78cb74bfc99eac78e: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:43.486014 1277 log.cc:824] T c69ca4c8880f4293a5b8afe11199634f P be2770f65a754de78cb74bfc99eac78e: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:43.486474 1279 tablet_bootstrap.cc:654] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:43.486481 1278 tablet_bootstrap.cc:654] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:43.487344 1279 log.cc:824] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:43.487344 1278 log.cc:824] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:43.488111 1277 tablet_bootstrap.cc:492] T c69ca4c8880f4293a5b8afe11199634f P be2770f65a754de78cb74bfc99eac78e: No bootstrap required, opened a new log
I20260806 18:33:43.488375 1277 ts_tablet_manager.cc:1397] T c69ca4c8880f4293a5b8afe11199634f P be2770f65a754de78cb74bfc99eac78e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:33:43.489282 1278 tablet_bootstrap.cc:492] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007: No bootstrap required, opened a new log
I20260806 18:33:43.489282 1279 tablet_bootstrap.cc:492] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2: No bootstrap required, opened a new log
I20260806 18:33:43.489468 1279 ts_tablet_manager.cc:1397] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:33:43.489487 1278 ts_tablet_manager.cc:1397] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:33:43.491725 1277 raft_consensus.cc:348] T c69ca4c8880f4293a5b8afe11199634f P be2770f65a754de78cb74bfc99eac78e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2770f65a754de78cb74bfc99eac78e" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39549 } } peers { permanent_uuid: "d8426646565a4a49b183817a98b603a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46031 } } peers { permanent_uuid: "010f090f5aad4992a456166d65056007" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 43175 } }
I20260806 18:33:43.491991 1277 raft_consensus.cc:374] T c69ca4c8880f4293a5b8afe11199634f P be2770f65a754de78cb74bfc99eac78e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:43.492055 1277 raft_consensus.cc:729] T c69ca4c8880f4293a5b8afe11199634f P be2770f65a754de78cb74bfc99eac78e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be2770f65a754de78cb74bfc99eac78e, State: Initialized, Role: FOLLOWER
I20260806 18:33:43.492486 1277 consensus_queue.cc:260] T c69ca4c8880f4293a5b8afe11199634f P be2770f65a754de78cb74bfc99eac78e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2770f65a754de78cb74bfc99eac78e" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39549 } } peers { permanent_uuid: "d8426646565a4a49b183817a98b603a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46031 } } peers { permanent_uuid: "010f090f5aad4992a456166d65056007" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 43175 } }
I20260806 18:33:43.493319 872 heartbeater.cc:503] Master 127.7.67.254:35057 was elected leader, sending a full tablet report...
I20260806 18:33:43.493494 1277 ts_tablet_manager.cc:1428] T c69ca4c8880f4293a5b8afe11199634f P be2770f65a754de78cb74bfc99eac78e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:33:43.493727 1278 raft_consensus.cc:348] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2770f65a754de78cb74bfc99eac78e" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39549 } } peers { permanent_uuid: "d8426646565a4a49b183817a98b603a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46031 } } peers { permanent_uuid: "010f090f5aad4992a456166d65056007" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 43175 } }
I20260806 18:33:43.493983 1278 raft_consensus.cc:374] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:43.494040 1278 raft_consensus.cc:729] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 010f090f5aad4992a456166d65056007, State: Initialized, Role: FOLLOWER
I20260806 18:33:43.494262 1279 raft_consensus.cc:348] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2770f65a754de78cb74bfc99eac78e" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39549 } } peers { permanent_uuid: "d8426646565a4a49b183817a98b603a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46031 } } peers { permanent_uuid: "010f090f5aad4992a456166d65056007" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 43175 } }
I20260806 18:33:43.494517 1279 raft_consensus.cc:374] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:43.494465 1278 consensus_queue.cc:260] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2770f65a754de78cb74bfc99eac78e" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39549 } } peers { permanent_uuid: "d8426646565a4a49b183817a98b603a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46031 } } peers { permanent_uuid: "010f090f5aad4992a456166d65056007" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 43175 } }
I20260806 18:33:43.494571 1279 raft_consensus.cc:729] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8426646565a4a49b183817a98b603a2, State: Initialized, Role: FOLLOWER
I20260806 18:33:43.495008 1279 consensus_queue.cc:260] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2770f65a754de78cb74bfc99eac78e" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39549 } } peers { permanent_uuid: "d8426646565a4a49b183817a98b603a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46031 } } peers { permanent_uuid: "010f090f5aad4992a456166d65056007" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 43175 } }
I20260806 18:33:43.495579 1278 ts_tablet_manager.cc:1428] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 18:33:43.495692 1252 heartbeater.cc:503] Master 127.7.67.254:35057 was elected leader, sending a full tablet report...
I20260806 18:33:43.495954 1279 ts_tablet_manager.cc:1428] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260806 18:33:43.496454 1068 heartbeater.cc:503] Master 127.7.67.254:35057 was elected leader, sending a full tablet report...
W20260806 18:33:43.534996 874 tablet.cc:2369] T c69ca4c8880f4293a5b8afe11199634f P be2770f65a754de78cb74bfc99eac78e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:33:43.679565 1253 tablet.cc:2369] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:33:43.686335 1288 raft_consensus.cc:482] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:33:43.686574 1288 raft_consensus.cc:504] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2770f65a754de78cb74bfc99eac78e" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39549 } } peers { permanent_uuid: "d8426646565a4a49b183817a98b603a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46031 } } peers { permanent_uuid: "010f090f5aad4992a456166d65056007" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 43175 } }
I20260806 18:33:43.687830 1288 leader_election.cc:288] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be2770f65a754de78cb74bfc99eac78e (127.7.67.193:39549), 010f090f5aad4992a456166d65056007 (127.7.67.195:43175)
I20260806 18:33:43.691376 1071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:43.688096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:39549 (local address 127.7.67.194:54523)
0806 18:33:43.688312 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:43.688328 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:43.688429 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:43.688639 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:43.688642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:43.688658 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:43.688789 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:43.688795 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.689495 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.689499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:43.690335 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:43.690345 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.690969 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.690973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:43.691070 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:43.691091 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:33:43.691155 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 18:33:43.691228 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"spinlock_wait_cycles":37120}
I20260806 18:33:43.691399 1268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:43.688265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:54523 (local address 127.7.67.193:39549)
0806 18:33:43.688427 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:43.688432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:43.688450 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:33:43.688510 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:43.688513 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:43.688555 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:43.688619 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:43.688890 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.689397 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.690469 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.690838 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.690972 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:43.691024 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:43.691177 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 18:33:43.691245 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 18:33:43.691689 1275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:43.688272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:43917 (local address 127.7.67.195:43175)
0806 18:33:43.688414 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:43.688419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:43.688668 (+ 249us) server_negotiation.cc:407] Connection header received
0806 18:33:43.688813 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:43.688816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:43.688859 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:43.688921 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:43.689179 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.689670 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.690693 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.691120 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.691226 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:43.691273 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:43.691481 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 18:33:43.691565 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:33:43.692047 1289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:43.688148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:43175 (local address 127.7.67.194:43917)
0806 18:33:43.688578 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:43.688589 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:43.688714 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:43.688953 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:43.688956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:43.688971 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:43.689085 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:43.689090 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.689777 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.689780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:43.690574 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:43.690583 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.691240 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.691244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:43.691337 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:43.691359 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:33:43.691826 (+ 467us) client_negotiation.cc:240] Negotiation successful
0806 18:33:43.691897 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"spinlock_wait_cycles":1280,"thread_start_us":136,"threads_started":1}
I20260806 18:33:43.692070 797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c69ca4c8880f4293a5b8afe11199634f" candidate_uuid: "d8426646565a4a49b183817a98b603a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be2770f65a754de78cb74bfc99eac78e" is_pre_election: true
I20260806 18:33:43.692415 797 raft_consensus.cc:2454] T c69ca4c8880f4293a5b8afe11199634f P be2770f65a754de78cb74bfc99eac78e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8426646565a4a49b183817a98b603a2 in term 0.
I20260806 18:33:43.692678 1192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c69ca4c8880f4293a5b8afe11199634f" candidate_uuid: "d8426646565a4a49b183817a98b603a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010f090f5aad4992a456166d65056007" is_pre_election: true
I20260806 18:33:43.692926 901 leader_election.cc:302] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be2770f65a754de78cb74bfc99eac78e, d8426646565a4a49b183817a98b603a2; no voters:
I20260806 18:33:43.692976 1192 raft_consensus.cc:2454] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8426646565a4a49b183817a98b603a2 in term 0.
I20260806 18:33:43.693186 1288 raft_consensus.cc:2781] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:33:43.693276 1288 raft_consensus.cc:482] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:33:43.693329 1288 raft_consensus.cc:3037] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:43.694358 1288 raft_consensus.cc:504] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2770f65a754de78cb74bfc99eac78e" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39549 } } peers { permanent_uuid: "d8426646565a4a49b183817a98b603a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46031 } } peers { permanent_uuid: "010f090f5aad4992a456166d65056007" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 43175 } }
I20260806 18:33:43.694697 1288 leader_election.cc:288] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [CANDIDATE]: Term 1 election: Requested vote from peers be2770f65a754de78cb74bfc99eac78e (127.7.67.193:39549), 010f090f5aad4992a456166d65056007 (127.7.67.195:43175)
I20260806 18:33:43.695062 797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c69ca4c8880f4293a5b8afe11199634f" candidate_uuid: "d8426646565a4a49b183817a98b603a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be2770f65a754de78cb74bfc99eac78e"
I20260806 18:33:43.695071 1192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c69ca4c8880f4293a5b8afe11199634f" candidate_uuid: "d8426646565a4a49b183817a98b603a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010f090f5aad4992a456166d65056007"
I20260806 18:33:43.695192 1192 raft_consensus.cc:3037] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:43.695192 797 raft_consensus.cc:3037] T c69ca4c8880f4293a5b8afe11199634f P be2770f65a754de78cb74bfc99eac78e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:43.696463 1192 raft_consensus.cc:2454] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8426646565a4a49b183817a98b603a2 in term 1.
I20260806 18:33:43.696470 797 raft_consensus.cc:2454] T c69ca4c8880f4293a5b8afe11199634f P be2770f65a754de78cb74bfc99eac78e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8426646565a4a49b183817a98b603a2 in term 1.
I20260806 18:33:43.696775 900 leader_election.cc:302] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010f090f5aad4992a456166d65056007, d8426646565a4a49b183817a98b603a2; no voters:
I20260806 18:33:43.696985 1288 raft_consensus.cc:2781] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:33:43.697170 1288 raft_consensus.cc:686] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [term 1 LEADER]: Becoming Leader. State: Replica: d8426646565a4a49b183817a98b603a2, State: Running, Role: LEADER
I20260806 18:33:43.697502 1288 consensus_queue.cc:237] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2770f65a754de78cb74bfc99eac78e" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39549 } } peers { permanent_uuid: "d8426646565a4a49b183817a98b603a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46031 } } peers { permanent_uuid: "010f090f5aad4992a456166d65056007" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 43175 } }
I20260806 18:33:43.700608 598 catalog_manager.cc:5495] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to d8426646565a4a49b183817a98b603a2 (127.7.67.194). New cstate: current_term: 1 leader_uuid: "d8426646565a4a49b183817a98b603a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2770f65a754de78cb74bfc99eac78e" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8426646565a4a49b183817a98b603a2" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 46031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "010f090f5aad4992a456166d65056007" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 43175 } health_report { overall_health: UNKNOWN } } }
W20260806 18:33:43.728215 1069 tablet.cc:2369] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:33:43.763540 1276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:43.760359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55808 (local address 127.7.67.194:46031)
0806 18:33:43.760503 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:43.760507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:43.760523 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:33:43.760607 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:43.760609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:43.760668 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:43.760771 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:33:43.760957 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.761471 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.762232 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:43.762445 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:43.762515 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:43.762748 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:33:43.763208 (+ 460us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:33:43.763296 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:43.763351 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 18:33:43.763402 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 18:33:43.769670 1192 raft_consensus.cc:1261] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007 [term 1 FOLLOWER]: Refusing update from remote peer d8426646565a4a49b183817a98b603a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:33:43.769702 797 raft_consensus.cc:1261] T c69ca4c8880f4293a5b8afe11199634f P be2770f65a754de78cb74bfc99eac78e [term 1 FOLLOWER]: Refusing update from remote peer d8426646565a4a49b183817a98b603a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:33:43.770434 1288 consensus_queue.cc:1035] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010f090f5aad4992a456166d65056007" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 43175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:33:43.770644 1290 consensus_queue.cc:1035] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be2770f65a754de78cb74bfc99eac78e" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:33:43.781529 1297 mvcc.cc:205] Tried to move back new op lower bound from 7315624852553269248 to 7315624852267724800. Current Snapshot: MvccSnapshot[applied={T|T < 7315624852553269248}]
I20260806 18:33:43.782981 1295 mvcc.cc:205] Tried to move back new op lower bound from 7315624852553269248 to 7315624852267724800. Current Snapshot: MvccSnapshot[applied={T|T < 7315624852553269248}]
I20260806 18:33:43.791061 597 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:33:43.791229 597 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:33:43.793752 597 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 783a31bd1f934e5092b4cecb322d2ba2: Sending DeleteTablet for 3 replicas of tablet c69ca4c8880f4293a5b8afe11199634f
I20260806 18:33:43.794703 764 tablet_service.cc:1514] Processing DeleteTablet for tablet c69ca4c8880f4293a5b8afe11199634f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:43 UTC) from {username='slave'} at 127.0.0.1:43188
I20260806 18:33:43.794693 1160 tablet_service.cc:1514] Processing DeleteTablet for tablet c69ca4c8880f4293a5b8afe11199634f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:43 UTC) from {username='slave'} at 127.0.0.1:35428
I20260806 18:33:43.794806 970 tablet_service.cc:1514] Processing DeleteTablet for tablet c69ca4c8880f4293a5b8afe11199634f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:43 UTC) from {username='slave'} at 127.0.0.1:55796
I20260806 18:33:43.795320 1310 tablet_replica.cc:330] T c69ca4c8880f4293a5b8afe11199634f P be2770f65a754de78cb74bfc99eac78e: stopping tablet replica
I20260806 18:33:43.795663 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 678
I20260806 18:33:43.802558 1312 tablet_replica.cc:330] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007: stopping tablet replica
I20260806 18:33:43.803025 1312 raft_consensus.cc:2229] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:33:43.803401 1312 raft_consensus.cc:2258] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:33:43.804692 1315 tablet_replica.cc:330] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2: stopping tablet replica
I20260806 18:33:43.805035 1315 raft_consensus.cc:2229] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:33:43.805496 1315 raft_consensus.cc:2258] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:33:43.805816 1312 ts_tablet_manager.cc:1905] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:33:43.807037 1315 ts_tablet_manager.cc:1905] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:33:43.808753 1312 ts_tablet_manager.cc:1918] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:33:43.808867 1312 log.cc:1196] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/c69ca4c8880f4293a5b8afe11199634f
I20260806 18:33:43.809170 1312 ts_tablet_manager.cc:1939] T c69ca4c8880f4293a5b8afe11199634f P 010f090f5aad4992a456166d65056007: Deleting consensus metadata
I20260806 18:33:43.809773 1315 ts_tablet_manager.cc:1918] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:33:43.809875 1315 log.cc:1196] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/c69ca4c8880f4293a5b8afe11199634f
I20260806 18:33:43.810197 1315 ts_tablet_manager.cc:1939] T c69ca4c8880f4293a5b8afe11199634f P d8426646565a4a49b183817a98b603a2: Deleting consensus metadata
I20260806 18:33:43.810638 575 catalog_manager.cc:4841] TS 010f090f5aad4992a456166d65056007 (127.7.67.195:43175): tablet c69ca4c8880f4293a5b8afe11199634f (table test-table [id=ae1f3bd434474e4cb573e799ce69c28d]) successfully deleted
I20260806 18:33:43.811352 576 catalog_manager.cc:4841] TS d8426646565a4a49b183817a98b603a2 (127.7.67.194:46031): tablet c69ca4c8880f4293a5b8afe11199634f (table test-table [id=ae1f3bd434474e4cb573e799ce69c28d]) successfully deleted
W20260806 18:33:43.811796 576 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.193:39549 (error 108)
I20260806 18:33:43.812014 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 878
I20260806 18:33:43.817665 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:43.814541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:39549 (local address 127.0.0.1:43200)
0806 18:33:43.817455 (+ 2914us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:43.817555 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:39549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2796}
W20260806 18:33:43.817853 576 catalog_manager.cc:4584] TS be2770f65a754de78cb74bfc99eac78e (127.7.67.193:39549): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c69ca4c8880f4293a5b8afe11199634f: Network error: Client connection negotiation failed: client connection to 127.7.67.193:39549: connect: Connection refused (error 111)
I20260806 18:33:43.823639 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 1072
I20260806 18:33:43.838248 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 556
2026-08-06T18:33:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1284 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:33:43 dist-test-slave-5bn2 krb5kdc[1321](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:33:43 dist-test-slave-5bn2 krb5kdc[1321](info): set up 2 sockets
Aug 06 18:33:43 dist-test-slave-5bn2 krb5kdc[1321](info): commencing operation
krb5kdc: starting...
W20260806 18:33:44.983111 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:33:45 dist-test-slave-5bn2 krb5kdc[1321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:33:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:33:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:33:45.111171 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:43533
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:43533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:33:45.223757 1345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:45.224028 1345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:45.224082 1345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:45.228051 1345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:33:45.228121 1345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:45.228147 1345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:45.228168 1345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:33:45.228184 1345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:33:45.233760 1345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:43533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:43533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:45.234918 1345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:45.235992 1345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:45.244526 1351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:45.244628 1345 server_base.cc:1034] running on GCE node
W20260806 18:33:45.244520 1353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:45.244522 1355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:45.245360 1345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:45.246459 1345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:45.247633 1345 hybrid_clock.cc:648] HybridClock initialized: now 1786041225247618 us; error 28 us; skew 500 ppm
Aug 06 18:33:45 dist-test-slave-5bn2 krb5kdc[1321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041225, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:45.250494 1345 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:33:45.251482 1345 webserver.cc:460] Webserver started at http://127.7.67.254:36299/ using document root <none> and password file <none>
I20260806 18:33:45.252050 1345 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:45.252120 1345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:45.252331 1345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:45.254668 1345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "b772a1c6803840e58f3261a9a8f055f3"
format_stamp: "Formatted at 2026-08-06 18:33:45 on dist-test-slave-5bn2"
server_key: "2755ef30a58e4ebe91412ea85181deb5"
server_key_iv: "14c67c5795e2e9427d61e0ed6e817b39"
server_key_version: "encryptionkey@0"
I20260806 18:33:45.255286 1345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "b772a1c6803840e58f3261a9a8f055f3"
format_stamp: "Formatted at 2026-08-06 18:33:45 on dist-test-slave-5bn2"
server_key: "2755ef30a58e4ebe91412ea85181deb5"
server_key_iv: "14c67c5795e2e9427d61e0ed6e817b39"
server_key_version: "encryptionkey@0"
I20260806 18:33:45.259227 1345 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:33:45.261749 1367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:45.263109 1345 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:33:45.263242 1345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "b772a1c6803840e58f3261a9a8f055f3"
format_stamp: "Formatted at 2026-08-06 18:33:45 on dist-test-slave-5bn2"
server_key: "2755ef30a58e4ebe91412ea85181deb5"
server_key_iv: "14c67c5795e2e9427d61e0ed6e817b39"
server_key_version: "encryptionkey@0"
I20260806 18:33:45.263355 1345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:45.286547 1345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:45.289752 1345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:45.289958 1345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:45.297200 1345 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:43533
I20260806 18:33:45.297226 1447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:43533 every 8 connection(s)
I20260806 18:33:45.298215 1345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:33:45.301163 1448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:45.307075 1448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3: Bootstrap starting.
I20260806 18:33:45.308445 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 1345
I20260806 18:33:45.308732 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:33:45.309072 7439 external_mini_cluster.cc:1428] Setting key 0d7fc51a8fa46494bb6b04827babf49f
I20260806 18:33:45.310335 1448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:45.311147 1448 log.cc:824] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:45.313330 1448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3: No bootstrap required, opened a new log
I20260806 18:33:45.317631 1448 raft_consensus.cc:348] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b772a1c6803840e58f3261a9a8f055f3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 43533 } }
I20260806 18:33:45.317945 1448 raft_consensus.cc:374] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:45.318012 1448 raft_consensus.cc:729] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b772a1c6803840e58f3261a9a8f055f3, State: Initialized, Role: FOLLOWER
Aug 06 18:33:45 dist-test-slave-5bn2 krb5kdc[1321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:33:45.318496 1448 consensus_queue.cc:260] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b772a1c6803840e58f3261a9a8f055f3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 43533 } }
I20260806 18:33:45.318639 1448 raft_consensus.cc:388] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:33:45.318703 1448 raft_consensus.cc:482] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:33:45.318799 1448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:45.319761 1448 raft_consensus.cc:504] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b772a1c6803840e58f3261a9a8f055f3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 43533 } }
I20260806 18:33:45.320101 1448 leader_election.cc:302] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b772a1c6803840e58f3261a9a8f055f3; no voters:
I20260806 18:33:45.320446 1448 leader_election.cc:288] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:33:45.320545 1453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:33:45.320719 1453 raft_consensus.cc:686] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3 [term 1 LEADER]: Becoming Leader. State: Replica: b772a1c6803840e58f3261a9a8f055f3, State: Running, Role: LEADER
I20260806 18:33:45.321103 1453 consensus_queue.cc:237] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b772a1c6803840e58f3261a9a8f055f3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 43533 } }
I20260806 18:33:45.321844 1448 sys_catalog.cc:564] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:33:45.323002 1454 sys_catalog.cc:455] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b772a1c6803840e58f3261a9a8f055f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b772a1c6803840e58f3261a9a8f055f3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 43533 } } }
I20260806 18:33:45.323133 1454 sys_catalog.cc:458] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3 [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:45.323503 1457 sys_catalog.cc:455] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b772a1c6803840e58f3261a9a8f055f3. Latest consensus state: current_term: 1 leader_uuid: "b772a1c6803840e58f3261a9a8f055f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b772a1c6803840e58f3261a9a8f055f3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 43533 } } }
I20260806 18:33:45.323576 1469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:33:45.323618 1457 sys_catalog.cc:458] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3 [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:45.324872 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:45.310391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49304 (local address 127.7.67.254:43533)
0806 18:33:45.310945 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:45.310964 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:45.311002 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:33:45.311692 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:45.311711 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:45.312063 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:45.312364 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:45.313270 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:45.314327 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:45.315524 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:45.316051 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:45.316183 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:45.319056 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:45.319062 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:45.319077 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:45.319107 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:45.320925 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:45.321456 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:45.321460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:45.321465 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:45.321583 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:45.321886 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:45.321890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:45.321892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:45.322434 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:45.323315 (+ 881us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:45.323491 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 18:33:45.323823 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":138,"threads_started":1}
I20260806 18:33:45.326910 1469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:33:45.332791 1469 catalog_manager.cc:1347] Generated new cluster ID: bb773474e2504de7ad0148e7757f8646
I20260806 18:33:45.332871 1469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:33:45.341995 1469 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:33:45.343004 1469 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:33:45.348011 1469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3: Generated new TSK 0
I20260806 18:33:45.348672 1469 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
I20260806 18:33:45.385181 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:43533
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:33:45.499104 1493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:45.499356 1493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:45.499405 1493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:45.503454 1493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:45.503513 1493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:45.503572 1493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:33:45.508832 1493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:43533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:45.509835 1493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:45.510913 1493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:45.520385 1500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:45.520391 1509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:45.520396 1499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:45.520526 1493 server_base.cc:1034] running on GCE node
I20260806 18:33:45.521051 1493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:45.521682 1493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:45.522833 1493 hybrid_clock.cc:648] HybridClock initialized: now 1786041225522816 us; error 30 us; skew 500 ppm
Aug 06 18:33:45 dist-test-slave-5bn2 krb5kdc[1321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041225, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:45.525632 1493 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:33:45.526515 1493 webserver.cc:460] Webserver started at http://127.7.67.193:34915/ using document root <none> and password file <none>
I20260806 18:33:45.527108 1493 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:45.527175 1493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:45.527382 1493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:45.529459 1493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "c4efafccc86d455fb96fcf062bd8ac28"
format_stamp: "Formatted at 2026-08-06 18:33:45 on dist-test-slave-5bn2"
server_key: "254d4b2d6e96d9de0b7acfdd188858d2"
server_key_iv: "99aa24676b0bdef58eeefec96032f1ad"
server_key_version: "encryptionkey@0"
I20260806 18:33:45.530014 1493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "c4efafccc86d455fb96fcf062bd8ac28"
format_stamp: "Formatted at 2026-08-06 18:33:45 on dist-test-slave-5bn2"
server_key: "254d4b2d6e96d9de0b7acfdd188858d2"
server_key_iv: "99aa24676b0bdef58eeefec96032f1ad"
server_key_version: "encryptionkey@0"
I20260806 18:33:45.533834 1493 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260806 18:33:45.536334 1519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:45.537524 1493 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:33:45.537636 1493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "c4efafccc86d455fb96fcf062bd8ac28"
format_stamp: "Formatted at 2026-08-06 18:33:45 on dist-test-slave-5bn2"
server_key: "254d4b2d6e96d9de0b7acfdd188858d2"
server_key_iv: "99aa24676b0bdef58eeefec96032f1ad"
server_key_version: "encryptionkey@0"
I20260806 18:33:45.537746 1493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:45.549119 1493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:45.552208 1493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:45.552384 1493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:45.552973 1493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:45.553800 1493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:45.553854 1493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:45.553894 1493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:45.553921 1493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:45.563692 1493 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:45811
I20260806 18:33:45.563719 1688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:45811 every 8 connection(s)
I20260806 18:33:45.564621 1493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:33:45.572417 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 1493
I20260806 18:33:45.572556 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:33:45.572862 7439 external_mini_cluster.cc:1428] Setting key 0f67610744bcf3f42150e5f732a272f8
Aug 06 18:33:45 dist-test-slave-5bn2 krb5kdc[1321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041225, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:33:45.579622 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:45.566509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:33327 (local address 127.7.67.254:43533)
0806 18:33:45.566669 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:45.566673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:45.567566 (+ 893us) server_negotiation.cc:407] Connection header received
0806 18:33:45.569026 (+ 1460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:45.569030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:45.569096 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:45.569183 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:45.570483 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:45.571091 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:45.572310 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:45.572587 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:45.572710 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:45.575735 (+ 3025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:45.575740 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:45.575743 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:45.575776 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:45.577287 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:45.577908 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:45.577911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:45.577912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:45.577989 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:45.578350 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:45.578352 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:45.578353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:45.578534 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:45.578620 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:45.579308 (+ 688us) server_negotiation.cc:299] Negotiation successful
0806 18:33:45.579440 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:33:45.580868 1692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:45.566901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:43533 (local address 127.7.67.193:33327)
0806 18:33:45.567414 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:45.567462 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:45.568796 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:45.569380 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:45.569387 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:45.569774 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:45.570309 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:45.570334 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:45.571227 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:45.571230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:45.572169 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:45.572183 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:45.572764 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:45.572769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:45.572943 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:45.573596 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:33:45.573619 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:45.575546 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:45.577417 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:45.577424 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:45.577451 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:45.577814 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:45.578070 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:45.578073 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:45.578075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:45.578268 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:45.578642 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:45.578649 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:45.579017 (+ 368us) client_negotiation.cc:769] Sending connection context
0806 18:33:45.579230 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 18:33:45.579599 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":109,"threads_started":1}
I20260806 18:33:45.582319 1689 heartbeater.cc:347] Connected to a master server at 127.7.67.254:43533
I20260806 18:33:45.582584 1689 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:45.583144 1689 heartbeater.cc:511] Master 127.7.67.254:43533 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.67.194@KRBTEST.COM; defaulting to no policy
I20260806 18:33:45.584798 1390 ts_manager.cc:194] Registered new tserver with Master: c4efafccc86d455fb96fcf062bd8ac28 (127.7.67.193:45811)
I20260806 18:33:45.585987 1390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:33327
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
I20260806 18:33:45.603076 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:43533
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:33:45.715727 1697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:45.716010 1697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:45.716068 1697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:45.720126 1697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:45.720196 1697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:45.720273 1697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:33:45.725716 1697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:43533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:45.726809 1697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:45.727869 1697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:33:45.737599 1697 server_base.cc:1034] running on GCE node
W20260806 18:33:45.737749 1705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:45.737767 1703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:45.737838 1707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:45.738237 1697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:45.738881 1697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:45.740067 1697 hybrid_clock.cc:648] HybridClock initialized: now 1786041225740048 us; error 32 us; skew 500 ppm
Aug 06 18:33:45 dist-test-slave-5bn2 krb5kdc[1321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041225, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:45.742937 1697 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:33:45.743793 1697 webserver.cc:460] Webserver started at http://127.7.67.194:46165/ using document root <none> and password file <none>
I20260806 18:33:45.744343 1697 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:45.744410 1697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:45.744606 1697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:45.746646 1697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "a3f2b8ecbc324b3498bba928811ea391"
format_stamp: "Formatted at 2026-08-06 18:33:45 on dist-test-slave-5bn2"
server_key: "21236b1e718a678483aea40a6068ff9a"
server_key_iv: "cf6831f2ceca9c6236365507380476eb"
server_key_version: "encryptionkey@0"
I20260806 18:33:45.747241 1697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "a3f2b8ecbc324b3498bba928811ea391"
format_stamp: "Formatted at 2026-08-06 18:33:45 on dist-test-slave-5bn2"
server_key: "21236b1e718a678483aea40a6068ff9a"
server_key_iv: "cf6831f2ceca9c6236365507380476eb"
server_key_version: "encryptionkey@0"
I20260806 18:33:45.751024 1697 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:33:45.753432 1724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:45.754593 1697 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:33:45.754698 1697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "a3f2b8ecbc324b3498bba928811ea391"
format_stamp: "Formatted at 2026-08-06 18:33:45 on dist-test-slave-5bn2"
server_key: "21236b1e718a678483aea40a6068ff9a"
server_key_iv: "cf6831f2ceca9c6236365507380476eb"
server_key_version: "encryptionkey@0"
I20260806 18:33:45.754804 1697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:45.768219 1697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:45.771144 1697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:45.771322 1697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:45.771927 1697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:45.772755 1697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:45.772802 1697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:45.772846 1697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:45.772876 1697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:45.781623 1697 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:38243
I20260806 18:33:45.781648 1896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:38243 every 8 connection(s)
I20260806 18:33:45.782519 1697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:33:45.789960 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 1697
I20260806 18:33:45.790114 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:33:45.790495 7439 external_mini_cluster.cc:1428] Setting key 0b0941345ba04daea9848e204a42d5b0
Aug 06 18:33:45 dist-test-slave-5bn2 krb5kdc[1321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041225, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:33:45.796464 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:45.784376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:33613 (local address 127.7.67.254:43533)
0806 18:33:45.784531 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:45.784536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:45.785394 (+ 858us) server_negotiation.cc:407] Connection header received
0806 18:33:45.786734 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:45.786738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:45.786790 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:45.786900 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:45.788070 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:45.788619 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:45.789735 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:45.789957 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:45.790046 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:45.792846 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:45.792850 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:45.792852 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:45.792876 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:45.794363 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:45.794844 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:45.794871 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:45.794874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:45.794970 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:45.795272 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:45.795275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:45.795276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:45.795517 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:45.795611 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:45.796235 (+ 624us) server_negotiation.cc:299] Negotiation successful
0806 18:33:45.796340 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:33:45.797596 1901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:45.784765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:43533 (local address 127.7.67.194:33613)
0806 18:33:45.785237 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:45.785281 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:45.786510 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:45.787055 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:45.787061 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:45.787418 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:45.787912 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:45.787927 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:45.788734 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:45.788736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:45.789615 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:45.789624 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:45.790127 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:45.790132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:45.790328 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:45.790953 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:33:45.790969 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:45.792673 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:45.794480 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:45.794485 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:45.794498 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:45.794751 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:45.795049 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:45.795051 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:45.795053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:45.795187 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:45.795636 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:45.795643 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:45.795967 (+ 324us) client_negotiation.cc:769] Sending connection context
0806 18:33:45.796177 (+ 210us) client_negotiation.cc:240] Negotiation successful
0806 18:33:45.796516 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":129,"threads_started":1}
I20260806 18:33:45.798774 1898 heartbeater.cc:347] Connected to a master server at 127.7.67.254:43533
I20260806 18:33:45.799050 1898 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:45.799572 1898 heartbeater.cc:511] Master 127.7.67.254:43533 requested a full tablet report, sending...
I20260806 18:33:45.800661 1390 ts_manager.cc:194] Registered new tserver with Master: a3f2b8ecbc324b3498bba928811ea391 (127.7.67.194:38243)
I20260806 18:33:45.801445 1390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:33613
WARNING: no policy specified for kudu/127.7.67.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
I20260806 18:33:45.819586 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:43533
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:33:45.939612 1906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:45.939885 1906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:45.939927 1906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:45.943987 1906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:45.944070 1906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:45.944159 1906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:33:45.949553 1906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:43533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:45.950675 1906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:45.951730 1906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:45.961727 1917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:45.961784 1913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:45.961737 1914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:45.961928 1906 server_base.cc:1034] running on GCE node
I20260806 18:33:45.962407 1906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:45.963127 1906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:45.964305 1906 hybrid_clock.cc:648] HybridClock initialized: now 1786041225964271 us; error 54 us; skew 500 ppm
Aug 06 18:33:45 dist-test-slave-5bn2 krb5kdc[1321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041225, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:45.967195 1906 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:33:45.968114 1906 webserver.cc:460] Webserver started at http://127.7.67.195:34605/ using document root <none> and password file <none>
I20260806 18:33:45.968690 1906 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:45.968751 1906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:45.968950 1906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:45.971109 1906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "3c1cb953de154a30b646d82efa264505"
format_stamp: "Formatted at 2026-08-06 18:33:45 on dist-test-slave-5bn2"
server_key: "9dacda00b4752a3f27c95f7750f3e414"
server_key_iv: "8f19567413083288f9bdd5f15f94bf10"
server_key_version: "encryptionkey@0"
I20260806 18:33:45.971683 1906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "3c1cb953de154a30b646d82efa264505"
format_stamp: "Formatted at 2026-08-06 18:33:45 on dist-test-slave-5bn2"
server_key: "9dacda00b4752a3f27c95f7750f3e414"
server_key_iv: "8f19567413083288f9bdd5f15f94bf10"
server_key_version: "encryptionkey@0"
I20260806 18:33:45.975489 1906 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:33:45.978027 1926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:45.979362 1906 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:33:45.979481 1906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "3c1cb953de154a30b646d82efa264505"
format_stamp: "Formatted at 2026-08-06 18:33:45 on dist-test-slave-5bn2"
server_key: "9dacda00b4752a3f27c95f7750f3e414"
server_key_iv: "8f19567413083288f9bdd5f15f94bf10"
server_key_version: "encryptionkey@0"
I20260806 18:33:45.979605 1906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:45.993417 1906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:45.996388 1906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:45.996575 1906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:45.997167 1906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:45.998008 1906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:45.998057 1906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:45.998098 1906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:45.998127 1906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:46.006768 1906 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:46349
I20260806 18:33:46.006791 2091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:46349 every 8 connection(s)
I20260806 18:33:46.007725 1906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
Aug 06 18:33:46 dist-test-slave-5bn2 krb5kdc[1321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041225, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:33:46.017570 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 1906
I20260806 18:33:46.017686 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:33:46.017944 7439 external_mini_cluster.cc:1428] Setting key b786f02a9e5f00150de3755d7ad9ce3e
I20260806 18:33:46.021561 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.009614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:58633 (local address 127.7.67.254:43533)
0806 18:33:46.009772 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:46.009777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:46.010654 (+ 877us) server_negotiation.cc:407] Connection header received
0806 18:33:46.012117 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:46.012120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:46.012173 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:46.012250 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:46.013434 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.013955 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.015127 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.015340 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.015420 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:46.017996 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:46.018001 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:46.018004 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:46.018035 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:46.019495 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:46.019996 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:46.020000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:46.020001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:46.020089 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:46.020411 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:46.020415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:46.020416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:46.020607 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:46.020693 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:46.021207 (+ 514us) server_negotiation.cc:299] Negotiation successful
0806 18:33:46.021345 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:33:46.022401 2095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.010003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:43533 (local address 127.7.67.195:58633)
0806 18:33:46.010492 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:46.010539 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:46.011908 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:46.012404 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:46.012411 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:46.012796 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:46.013307 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:46.013322 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.014043 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.014045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:46.015020 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:46.015030 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.015509 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.015513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:46.015682 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:46.016255 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:33:46.016272 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:46.017813 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:46.019617 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:46.019623 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:46.019640 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:46.019899 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:46.020174 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:46.020177 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:46.020179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:46.020322 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:46.020710 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:46.020717 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:46.020971 (+ 254us) client_negotiation.cc:769] Sending connection context
0806 18:33:46.021142 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 18:33:46.021503 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":115,"threads_started":1}
I20260806 18:33:46.023476 2092 heartbeater.cc:347] Connected to a master server at 127.7.67.254:43533
I20260806 18:33:46.023725 2092 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:46.024240 2092 heartbeater.cc:511] Master 127.7.67.254:43533 requested a full tablet report, sending...
I20260806 18:33:46.025265 1390 ts_manager.cc:194] Registered new tserver with Master: 3c1cb953de154a30b646d82efa264505 (127.7.67.195:46349)
I20260806 18:33:46.025818 1390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:58633
I20260806 18:33:46.031790 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:33:46.041234 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.033740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49310 (local address 127.7.67.254:43533)
0806 18:33:46.033871 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:46.033876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:46.034052 (+ 176us) server_negotiation.cc:407] Connection header received
0806 18:33:46.034200 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:46.034202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:46.034242 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:46.034302 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:46.035030 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.035539 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.036362 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.036575 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.036661 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:46.038192 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:46.038196 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:46.038198 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:46.038222 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:46.039477 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:46.039986 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:46.039989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:46.039990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:46.040066 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:46.040375 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:46.040378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:46.040379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:46.040593 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:46.040728 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:46.040998 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 18:33:46.041088 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 18:33:46.044406 1390 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49310:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:33:46.047515 1390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:33:46.062453 2110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.056341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:45811 (local address 127.0.0.1:40212)
0806 18:33:46.058138 (+ 1797us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:46.058151 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:46.058241 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:46.058828 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:46.058832 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:46.058885 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:46.058972 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:46.058977 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.060251 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.060254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:46.061101 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:46.061112 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.061877 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.061881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:46.062002 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:46.062042 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 18:33:46.062150 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 18:33:46.062277 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1737,"mutex_wait_us":23,"spinlock_wait_cycles":3968,"thread_start_us":86,"threads_started":1}
I20260806 18:33:46.062458 2118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.056864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40212 (local address 127.7.67.193:45811)
0806 18:33:46.057249 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:46.057266 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:46.058352 (+ 1086us) server_negotiation.cc:407] Connection header received
0806 18:33:46.058432 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:46.058436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:46.058609 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:46.058753 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:46.059091 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.060111 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.061221 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.061759 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.061864 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:46.062026 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:46.062189 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 18:33:46.062285 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":93,"threads_started":1}
I20260806 18:33:46.062671 2115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.056050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:46349 (local address 127.0.0.1:56166)
0806 18:33:46.057116 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:46.057170 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:46.057309 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:46.057907 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:46.057913 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:46.057936 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:46.058082 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:46.058091 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.059533 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.059536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:46.061137 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:46.061149 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.062252 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.062256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:46.062343 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:46.062365 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:33:46.062414 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 18:33:46.062517 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"mutex_wait_us":86,"thread_start_us":95,"threads_started":1}
I20260806 18:33:46.063073 2111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.056048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:38243 (local address 127.0.0.1:55214)
0806 18:33:46.057535 (+ 1487us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:46.057546 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:46.057616 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:46.058199 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:46.058202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:46.058232 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:46.058327 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:46.058333 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.059590 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.059593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:46.061138 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:46.061149 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.062414 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.062418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:46.062662 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:46.062685 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:33:46.062829 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 18:33:46.062936 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1425,"mutex_wait_us":211,"spinlock_wait_cycles":2304,"thread_start_us":87,"threads_started":1}
I20260806 18:33:46.063406 2119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.056906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56166 (local address 127.7.67.195:46349)
0806 18:33:46.057428 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:46.057441 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:46.057466 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:33:46.057548 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:46.057553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:46.057748 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:46.057897 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:46.058218 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.059388 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.061283 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.062110 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.062976 (+ 866us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:46.063117 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:46.063173 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 18:33:46.063260 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":176,"threads_started":1}
I20260806 18:33:46.063752 2114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.056217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55214 (local address 127.7.67.194:38243)
0806 18:33:46.057838 (+ 1621us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:46.057848 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:46.057866 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:33:46.057926 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:46.057930 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:46.058074 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:46.058198 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:46.058450 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.059412 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.061283 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.062110 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.062217 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:46.062355 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:46.063422 (+ 1067us) server_negotiation.cc:299] Negotiation successful
0806 18:33:46.063602 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1531,"thread_start_us":175,"threads_started":1}
I20260806 18:33:46.065920 1993 tablet_service.cc:1467] Processing CreateTablet for tablet 2bbed5243fab4687b1e65fca32a6cf84 (DEFAULT_TABLE table=test-table [id=b72e747a24db4a6f89dd1b44d8e8336b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:46.065924 1585 tablet_service.cc:1467] Processing CreateTablet for tablet 2bbed5243fab4687b1e65fca32a6cf84 (DEFAULT_TABLE table=test-table [id=b72e747a24db4a6f89dd1b44d8e8336b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:46.066366 1792 tablet_service.cc:1467] Processing CreateTablet for tablet 2bbed5243fab4687b1e65fca32a6cf84 (DEFAULT_TABLE table=test-table [id=b72e747a24db4a6f89dd1b44d8e8336b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:46.066958 1993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bbed5243fab4687b1e65fca32a6cf84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:46.067039 1585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bbed5243fab4687b1e65fca32a6cf84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:46.067322 1792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bbed5243fab4687b1e65fca32a6cf84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:46.073377 2120 tablet_bootstrap.cc:492] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505: Bootstrap starting.
I20260806 18:33:46.074831 2121 tablet_bootstrap.cc:492] T 2bbed5243fab4687b1e65fca32a6cf84 P a3f2b8ecbc324b3498bba928811ea391: Bootstrap starting.
I20260806 18:33:46.075676 2120 tablet_bootstrap.cc:654] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:46.076742 2121 tablet_bootstrap.cc:654] T 2bbed5243fab4687b1e65fca32a6cf84 P a3f2b8ecbc324b3498bba928811ea391: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:46.076930 2120 log.cc:824] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:46.077829 2121 log.cc:824] T 2bbed5243fab4687b1e65fca32a6cf84 P a3f2b8ecbc324b3498bba928811ea391: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:46.078428 2122 tablet_bootstrap.cc:492] T 2bbed5243fab4687b1e65fca32a6cf84 P c4efafccc86d455fb96fcf062bd8ac28: Bootstrap starting.
I20260806 18:33:46.078895 2120 tablet_bootstrap.cc:492] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505: No bootstrap required, opened a new log
I20260806 18:33:46.079088 2120 ts_tablet_manager.cc:1397] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:33:46.080183 2121 tablet_bootstrap.cc:492] T 2bbed5243fab4687b1e65fca32a6cf84 P a3f2b8ecbc324b3498bba928811ea391: No bootstrap required, opened a new log
I20260806 18:33:46.080459 2121 ts_tablet_manager.cc:1397] T 2bbed5243fab4687b1e65fca32a6cf84 P a3f2b8ecbc324b3498bba928811ea391: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260806 18:33:46.080544 2122 tablet_bootstrap.cc:654] T 2bbed5243fab4687b1e65fca32a6cf84 P c4efafccc86d455fb96fcf062bd8ac28: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:46.081817 2122 log.cc:824] T 2bbed5243fab4687b1e65fca32a6cf84 P c4efafccc86d455fb96fcf062bd8ac28: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:46.082406 2120 raft_consensus.cc:348] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f2b8ecbc324b3498bba928811ea391" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 38243 } } peers { permanent_uuid: "3c1cb953de154a30b646d82efa264505" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46349 } } peers { permanent_uuid: "c4efafccc86d455fb96fcf062bd8ac28" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45811 } }
I20260806 18:33:46.082693 2120 raft_consensus.cc:374] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:46.082765 2120 raft_consensus.cc:729] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c1cb953de154a30b646d82efa264505, State: Initialized, Role: FOLLOWER
I20260806 18:33:46.083328 2120 consensus_queue.cc:260] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f2b8ecbc324b3498bba928811ea391" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 38243 } } peers { permanent_uuid: "3c1cb953de154a30b646d82efa264505" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46349 } } peers { permanent_uuid: "c4efafccc86d455fb96fcf062bd8ac28" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45811 } }
I20260806 18:33:46.084075 2092 heartbeater.cc:503] Master 127.7.67.254:43533 was elected leader, sending a full tablet report...
I20260806 18:33:46.084333 2120 ts_tablet_manager.cc:1428] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:33:46.084448 2121 raft_consensus.cc:348] T 2bbed5243fab4687b1e65fca32a6cf84 P a3f2b8ecbc324b3498bba928811ea391 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f2b8ecbc324b3498bba928811ea391" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 38243 } } peers { permanent_uuid: "3c1cb953de154a30b646d82efa264505" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46349 } } peers { permanent_uuid: "c4efafccc86d455fb96fcf062bd8ac28" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45811 } }
I20260806 18:33:46.084690 2121 raft_consensus.cc:374] T 2bbed5243fab4687b1e65fca32a6cf84 P a3f2b8ecbc324b3498bba928811ea391 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:46.084756 2121 raft_consensus.cc:729] T 2bbed5243fab4687b1e65fca32a6cf84 P a3f2b8ecbc324b3498bba928811ea391 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3f2b8ecbc324b3498bba928811ea391, State: Initialized, Role: FOLLOWER
I20260806 18:33:46.085187 2121 consensus_queue.cc:260] T 2bbed5243fab4687b1e65fca32a6cf84 P a3f2b8ecbc324b3498bba928811ea391 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f2b8ecbc324b3498bba928811ea391" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 38243 } } peers { permanent_uuid: "3c1cb953de154a30b646d82efa264505" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46349 } } peers { permanent_uuid: "c4efafccc86d455fb96fcf062bd8ac28" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45811 } }
I20260806 18:33:46.085816 1898 heartbeater.cc:503] Master 127.7.67.254:43533 was elected leader, sending a full tablet report...
I20260806 18:33:46.086156 2121 ts_tablet_manager.cc:1428] T 2bbed5243fab4687b1e65fca32a6cf84 P a3f2b8ecbc324b3498bba928811ea391: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:33:46.086364 2122 tablet_bootstrap.cc:492] T 2bbed5243fab4687b1e65fca32a6cf84 P c4efafccc86d455fb96fcf062bd8ac28: No bootstrap required, opened a new log
I20260806 18:33:46.086593 2122 ts_tablet_manager.cc:1397] T 2bbed5243fab4687b1e65fca32a6cf84 P c4efafccc86d455fb96fcf062bd8ac28: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 18:33:46.090269 2122 raft_consensus.cc:348] T 2bbed5243fab4687b1e65fca32a6cf84 P c4efafccc86d455fb96fcf062bd8ac28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f2b8ecbc324b3498bba928811ea391" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 38243 } } peers { permanent_uuid: "3c1cb953de154a30b646d82efa264505" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46349 } } peers { permanent_uuid: "c4efafccc86d455fb96fcf062bd8ac28" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45811 } }
I20260806 18:33:46.090490 2122 raft_consensus.cc:374] T 2bbed5243fab4687b1e65fca32a6cf84 P c4efafccc86d455fb96fcf062bd8ac28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:46.090536 2122 raft_consensus.cc:729] T 2bbed5243fab4687b1e65fca32a6cf84 P c4efafccc86d455fb96fcf062bd8ac28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4efafccc86d455fb96fcf062bd8ac28, State: Initialized, Role: FOLLOWER
I20260806 18:33:46.090909 2122 consensus_queue.cc:260] T 2bbed5243fab4687b1e65fca32a6cf84 P c4efafccc86d455fb96fcf062bd8ac28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f2b8ecbc324b3498bba928811ea391" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 38243 } } peers { permanent_uuid: "3c1cb953de154a30b646d82efa264505" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46349 } } peers { permanent_uuid: "c4efafccc86d455fb96fcf062bd8ac28" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45811 } }
I20260806 18:33:46.091540 1689 heartbeater.cc:503] Master 127.7.67.254:43533 was elected leader, sending a full tablet report...
I20260806 18:33:46.091825 2122 ts_tablet_manager.cc:1428] T 2bbed5243fab4687b1e65fca32a6cf84 P c4efafccc86d455fb96fcf062bd8ac28: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:33:46.206743 2128 raft_consensus.cc:482] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:33:46.207016 2128 raft_consensus.cc:504] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f2b8ecbc324b3498bba928811ea391" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 38243 } } peers { permanent_uuid: "3c1cb953de154a30b646d82efa264505" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46349 } } peers { permanent_uuid: "c4efafccc86d455fb96fcf062bd8ac28" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45811 } }
I20260806 18:33:46.208216 2128 leader_election.cc:288] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3f2b8ecbc324b3498bba928811ea391 (127.7.67.194:38243), c4efafccc86d455fb96fcf062bd8ac28 (127.7.67.193:45811)
I20260806 18:33:46.212757 2095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.208513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:38243 (local address 127.7.67.195:33459)
0806 18:33:46.208667 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:46.208683 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:46.208787 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:46.209083 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:46.209086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:46.209108 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:46.209237 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:46.209244 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.210225 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.210227 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:46.211518 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:46.211530 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.212314 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.212318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:46.212440 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:46.212479 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 18:33:46.212537 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 18:33:46.212600 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":5}
I20260806 18:33:46.212879 2114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.208637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:33459 (local address 127.7.67.194:38243)
0806 18:33:46.208754 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:46.208759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:46.208776 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:33:46.208904 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:46.208907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:46.208963 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:46.209050 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:46.209344 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.210120 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.211651 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.212204 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.212311 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:46.212365 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:46.212663 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 18:33:46.212746 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 18:33:46.212889 2118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.208634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:51129 (local address 127.7.67.193:45811)
0806 18:33:46.208752 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:46.208757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:46.209180 (+ 423us) server_negotiation.cc:407] Connection header received
0806 18:33:46.209245 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:46.209247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:46.209287 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:46.209362 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:46.209591 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.210300 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.211695 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.212190 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.212310 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:46.212365 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:46.212706 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 18:33:46.212780 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:33:46.213245 2132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.208520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:45811 (local address 127.7.67.195:51129)
0806 18:33:46.209072 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:46.209085 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:46.209173 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:46.209380 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:46.209382 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:46.209396 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:46.209491 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:46.209495 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.210388 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.210391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:46.211558 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:46.211566 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.212311 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.212315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:46.212422 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:46.212519 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 18:33:46.212673 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 18:33:46.212746 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":232,"thread_start_us":157,"threads_started":1}
I20260806 18:33:46.213534 1827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bbed5243fab4687b1e65fca32a6cf84" candidate_uuid: "3c1cb953de154a30b646d82efa264505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3f2b8ecbc324b3498bba928811ea391" is_pre_election: true
I20260806 18:33:46.213840 1827 raft_consensus.cc:2454] T 2bbed5243fab4687b1e65fca32a6cf84 P a3f2b8ecbc324b3498bba928811ea391 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c1cb953de154a30b646d82efa264505 in term 0.
I20260806 18:33:46.213869 1612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bbed5243fab4687b1e65fca32a6cf84" candidate_uuid: "3c1cb953de154a30b646d82efa264505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4efafccc86d455fb96fcf062bd8ac28" is_pre_election: true
I20260806 18:33:46.214144 1612 raft_consensus.cc:2454] T 2bbed5243fab4687b1e65fca32a6cf84 P c4efafccc86d455fb96fcf062bd8ac28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c1cb953de154a30b646d82efa264505 in term 0.
I20260806 18:33:46.214330 1928 leader_election.cc:302] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c1cb953de154a30b646d82efa264505, a3f2b8ecbc324b3498bba928811ea391; no voters:
I20260806 18:33:46.214622 2128 raft_consensus.cc:2781] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:33:46.214705 2128 raft_consensus.cc:482] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:33:46.214735 2128 raft_consensus.cc:3037] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:46.215790 2128 raft_consensus.cc:504] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f2b8ecbc324b3498bba928811ea391" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 38243 } } peers { permanent_uuid: "3c1cb953de154a30b646d82efa264505" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46349 } } peers { permanent_uuid: "c4efafccc86d455fb96fcf062bd8ac28" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45811 } }
I20260806 18:33:46.216117 2128 leader_election.cc:288] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [CANDIDATE]: Term 1 election: Requested vote from peers a3f2b8ecbc324b3498bba928811ea391 (127.7.67.194:38243), c4efafccc86d455fb96fcf062bd8ac28 (127.7.67.193:45811)
I20260806 18:33:46.216466 1612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bbed5243fab4687b1e65fca32a6cf84" candidate_uuid: "3c1cb953de154a30b646d82efa264505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4efafccc86d455fb96fcf062bd8ac28"
I20260806 18:33:46.216486 1827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bbed5243fab4687b1e65fca32a6cf84" candidate_uuid: "3c1cb953de154a30b646d82efa264505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3f2b8ecbc324b3498bba928811ea391"
I20260806 18:33:46.216596 1612 raft_consensus.cc:3037] T 2bbed5243fab4687b1e65fca32a6cf84 P c4efafccc86d455fb96fcf062bd8ac28 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:46.216596 1827 raft_consensus.cc:3037] T 2bbed5243fab4687b1e65fca32a6cf84 P a3f2b8ecbc324b3498bba928811ea391 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:46.217859 1827 raft_consensus.cc:2454] T 2bbed5243fab4687b1e65fca32a6cf84 P a3f2b8ecbc324b3498bba928811ea391 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c1cb953de154a30b646d82efa264505 in term 1.
I20260806 18:33:46.217859 1612 raft_consensus.cc:2454] T 2bbed5243fab4687b1e65fca32a6cf84 P c4efafccc86d455fb96fcf062bd8ac28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c1cb953de154a30b646d82efa264505 in term 1.
I20260806 18:33:46.218165 1928 leader_election.cc:302] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3c1cb953de154a30b646d82efa264505, a3f2b8ecbc324b3498bba928811ea391, c4efafccc86d455fb96fcf062bd8ac28; no voters:
I20260806 18:33:46.218338 2128 raft_consensus.cc:2781] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:33:46.218570 2128 raft_consensus.cc:686] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [term 1 LEADER]: Becoming Leader. State: Replica: 3c1cb953de154a30b646d82efa264505, State: Running, Role: LEADER
I20260806 18:33:46.218918 2128 consensus_queue.cc:237] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f2b8ecbc324b3498bba928811ea391" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 38243 } } peers { permanent_uuid: "3c1cb953de154a30b646d82efa264505" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46349 } } peers { permanent_uuid: "c4efafccc86d455fb96fcf062bd8ac28" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45811 } }
I20260806 18:33:46.222028 1390 catalog_manager.cc:5495] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c1cb953de154a30b646d82efa264505 (127.7.67.195). New cstate: current_term: 1 leader_uuid: "3c1cb953de154a30b646d82efa264505" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f2b8ecbc324b3498bba928811ea391" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 38243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c1cb953de154a30b646d82efa264505" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 46349 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4efafccc86d455fb96fcf062bd8ac28" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45811 } health_report { overall_health: UNKNOWN } } }
I20260806 18:33:46.257089 2119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.253826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56178 (local address 127.7.67.195:46349)
0806 18:33:46.253946 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:46.253950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:46.253963 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:33:46.254059 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:46.254061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:46.254119 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:46.254209 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:33:46.254477 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.255034 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.255816 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.256022 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.256092 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:46.256327 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:33:46.256736 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:33:46.256841 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:46.256905 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 18:33:46.256947 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 18:33:46.258728 2094 tablet.cc:2369] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:33:46.263218 1827 raft_consensus.cc:1261] T 2bbed5243fab4687b1e65fca32a6cf84 P a3f2b8ecbc324b3498bba928811ea391 [term 1 FOLLOWER]: Refusing update from remote peer 3c1cb953de154a30b646d82efa264505: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:33:46.263219 1612 raft_consensus.cc:1261] T 2bbed5243fab4687b1e65fca32a6cf84 P c4efafccc86d455fb96fcf062bd8ac28 [term 1 FOLLOWER]: Refusing update from remote peer 3c1cb953de154a30b646d82efa264505: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:33:46.263927 2133 consensus_queue.cc:1035] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4efafccc86d455fb96fcf062bd8ac28" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 45811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:33:46.264104 2128 consensus_queue.cc:1035] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3f2b8ecbc324b3498bba928811ea391" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 38243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:33:46.270491 2139 mvcc.cc:205] Tried to move back new op lower bound from 7315624862766477312 to 7315624862595223552. Current Snapshot: MvccSnapshot[applied={T|T < 7315624862766477312}]
I20260806 18:33:46.271545 2140 mvcc.cc:205] Tried to move back new op lower bound from 7315624862766477312 to 7315624862595223552. Current Snapshot: MvccSnapshot[applied={T|T < 7315624862766477312}]
I20260806 18:33:46.276255 2141 mvcc.cc:205] Tried to move back new op lower bound from 7315624862766477312 to 7315624862595223552. Current Snapshot: MvccSnapshot[applied={T|T < 7315624862766477312}]
I20260806 18:33:46.280230 1389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:33:46.280423 1389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:33:46.283054 1389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b772a1c6803840e58f3261a9a8f055f3: Sending DeleteTablet for 3 replicas of tablet 2bbed5243fab4687b1e65fca32a6cf84
I20260806 18:33:46.283878 1792 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bbed5243fab4687b1e65fca32a6cf84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:46 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:55214
I20260806 18:33:46.283882 1993 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bbed5243fab4687b1e65fca32a6cf84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:46 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:56166
I20260806 18:33:46.284394 2151 tablet_replica.cc:330] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505: stopping tablet replica
I20260806 18:33:46.284761 2151 raft_consensus.cc:2229] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:33:46.284880 1585 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bbed5243fab4687b1e65fca32a6cf84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:46 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:40212
I20260806 18:33:46.285218 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 1493
I20260806 18:33:46.292461 2151 raft_consensus.cc:2258] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:33:46.294572 2151 ts_tablet_manager.cc:1905] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:33:46.294999 2153 tablet_replica.cc:330] T 2bbed5243fab4687b1e65fca32a6cf84 P a3f2b8ecbc324b3498bba928811ea391: stopping tablet replica
I20260806 18:33:46.295477 2153 raft_consensus.cc:2229] T 2bbed5243fab4687b1e65fca32a6cf84 P a3f2b8ecbc324b3498bba928811ea391 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:33:46.295950 2153 raft_consensus.cc:2258] T 2bbed5243fab4687b1e65fca32a6cf84 P a3f2b8ecbc324b3498bba928811ea391 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:33:46.299948 2153 ts_tablet_manager.cc:1905] T 2bbed5243fab4687b1e65fca32a6cf84 P a3f2b8ecbc324b3498bba928811ea391: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 18:33:46.304631 1368 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.193:45811 (error 108)
I20260806 18:33:46.305068 2151 ts_tablet_manager.cc:1918] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:33:46.305188 2151 log.cc:1196] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/2bbed5243fab4687b1e65fca32a6cf84
I20260806 18:33:46.305532 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 1697
I20260806 18:33:46.305554 2151 ts_tablet_manager.cc:1939] T 2bbed5243fab4687b1e65fca32a6cf84 P 3c1cb953de154a30b646d82efa264505: Deleting consensus metadata
I20260806 18:33:46.307894 1371 catalog_manager.cc:4841] TS 3c1cb953de154a30b646d82efa264505 (127.7.67.195:46349): tablet 2bbed5243fab4687b1e65fca32a6cf84 (table test-table [id=b72e747a24db4a6f89dd1b44d8e8336b]) successfully deleted
I20260806 18:33:46.315776 2111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.315417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:45811 (local address 127.0.0.1:40216)
0806 18:33:46.315577 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:46.315673 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 18:33:46.316010 1368 catalog_manager.cc:4584] TS c4efafccc86d455fb96fcf062bd8ac28 (127.7.67.193:45811): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2bbed5243fab4687b1e65fca32a6cf84: Network error: Client connection negotiation failed: client connection to 127.7.67.193:45811: connect: Connection refused (error 111)
I20260806 18:33:46.318526 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 1906
I20260806 18:33:46.319172 2111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.318804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:38243 (local address 127.0.0.1:55222)
0806 18:33:46.319022 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:46.319088 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.194:38243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260806 18:33:46.331171 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 1345
2026-08-06T18:33:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2495 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T18:33:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:33:46Z Disabled control of system clock
I20260806 18:33:46.368144 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:38925
--webserver_interface=127.7.67.254
--webserver_port=0
--builtin_ntp_servers=127.7.67.212:35195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:38925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:33:46.482054 2160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:46.482344 2160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:46.482383 2160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:46.486537 2160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:33:46.486610 2160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:46.486624 2160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:46.486642 2160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:33:46.486656 2160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:33:46.491677 2160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:35195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:38925
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:38925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:46.492796 2160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:46.494272 2160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:46.502985 2172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:46.503023 2160 server_base.cc:1034] running on GCE node
W20260806 18:33:46.502990 2174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:46.503044 2171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:46.503834 2160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:46.504920 2160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:46.506103 2160 hybrid_clock.cc:648] HybridClock initialized: now 1786041226506088 us; error 28 us; skew 500 ppm
I20260806 18:33:46.508091 2160 webserver.cc:460] Webserver started at http://127.7.67.254:40971/ using document root <none> and password file <none>
I20260806 18:33:46.508677 2160 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:46.508744 2160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:46.508960 2160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:46.511134 2160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "a9fd47a75f9a4bb8b6e4f03285177460"
format_stamp: "Formatted at 2026-08-06 18:33:46 on dist-test-slave-5bn2"
server_key: "e176a8ca3c50643781361e04f5ec124f"
server_key_iv: "1b6b3f2a8b84c2d73dea7dcef8bba2b2"
server_key_version: "encryptionkey@0"
I20260806 18:33:46.511706 2160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "a9fd47a75f9a4bb8b6e4f03285177460"
format_stamp: "Formatted at 2026-08-06 18:33:46 on dist-test-slave-5bn2"
server_key: "e176a8ca3c50643781361e04f5ec124f"
server_key_iv: "1b6b3f2a8b84c2d73dea7dcef8bba2b2"
server_key_version: "encryptionkey@0"
I20260806 18:33:46.515483 2160 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:33:46.517853 2183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:46.519133 2160 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:33:46.519250 2160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "a9fd47a75f9a4bb8b6e4f03285177460"
format_stamp: "Formatted at 2026-08-06 18:33:46 on dist-test-slave-5bn2"
server_key: "e176a8ca3c50643781361e04f5ec124f"
server_key_iv: "1b6b3f2a8b84c2d73dea7dcef8bba2b2"
server_key_version: "encryptionkey@0"
I20260806 18:33:46.519356 2160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:46.542114 2160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:46.542953 2160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:46.543143 2160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:46.550195 2160 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:38925
I20260806 18:33:46.550235 2258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:38925 every 8 connection(s)
I20260806 18:33:46.551218 2160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:33:46.554283 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 2160
I20260806 18:33:46.554409 2259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:46.554421 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:33:46.554737 7439 external_mini_cluster.cc:1428] Setting key cb5c82e0167a4e1dab1c342edfc63865
I20260806 18:33:46.561450 2259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460: Bootstrap starting.
I20260806 18:33:46.564255 2259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:46.565011 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.556298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35630 (local address 127.7.67.254:38925)
0806 18:33:46.557007 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:46.557023 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:46.557059 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:33:46.557782 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:46.557800 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:46.558244 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:46.558578 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:46.559068 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.560164 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.561354 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.562091 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.562252 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:46.563195 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:46.563203 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:46.563221 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:46.563646 (+ 425us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:46.563705 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:46.563727 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:46.563818 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:46.563952 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 18:33:46.564225 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":167,"threads_started":1}
I20260806 18:33:46.565024 2259 log.cc:824] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:46.567343 2259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460: No bootstrap required, opened a new log
I20260806 18:33:46.571094 2259 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9fd47a75f9a4bb8b6e4f03285177460" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 38925 } }
I20260806 18:33:46.571395 2259 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:46.571457 2259 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9fd47a75f9a4bb8b6e4f03285177460, State: Initialized, Role: FOLLOWER
I20260806 18:33:46.571864 2259 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9fd47a75f9a4bb8b6e4f03285177460" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 38925 } }
I20260806 18:33:46.572010 2259 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:33:46.572079 2259 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:33:46.572180 2259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:46.573156 2259 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9fd47a75f9a4bb8b6e4f03285177460" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 38925 } }
I20260806 18:33:46.573489 2259 leader_election.cc:302] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9fd47a75f9a4bb8b6e4f03285177460; no voters:
I20260806 18:33:46.573792 2259 leader_election.cc:288] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:33:46.573920 2267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:33:46.574129 2267 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460 [term 1 LEADER]: Becoming Leader. State: Replica: a9fd47a75f9a4bb8b6e4f03285177460, State: Running, Role: LEADER
I20260806 18:33:46.574440 2267 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9fd47a75f9a4bb8b6e4f03285177460" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 38925 } }
I20260806 18:33:46.575136 2259 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:33:46.576198 2268 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9fd47a75f9a4bb8b6e4f03285177460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9fd47a75f9a4bb8b6e4f03285177460" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 38925 } } }
I20260806 18:33:46.576224 2269 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9fd47a75f9a4bb8b6e4f03285177460. Latest consensus state: current_term: 1 leader_uuid: "a9fd47a75f9a4bb8b6e4f03285177460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9fd47a75f9a4bb8b6e4f03285177460" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 38925 } } }
I20260806 18:33:46.576335 2268 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460 [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:46.576359 2269 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460 [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:46.576687 2278 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:33:46.579982 2278 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:33:46.585382 2278 catalog_manager.cc:1347] Generated new cluster ID: 5beb8babb02f402880de12cc8de8b47a
I20260806 18:33:46.585464 2278 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:33:46.597349 2278 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:33:46.598630 2278 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:33:46.604809 2278 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460: Generated new TSK 0
I20260806 18:33:46.605646 2278 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:33:46.609894 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:38925
--builtin_ntp_servers=127.7.67.212:35195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:33:46.724781 2293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:46.725051 2293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:46.725095 2293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:46.729125 2293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:46.729188 2293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:46.729256 2293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:33:46.734033 2293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:35195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:38925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:46.735198 2293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:46.736603 2293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:46.746084 2301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:46.746084 2300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:46.746104 2305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:46.746476 2293 server_base.cc:1034] running on GCE node
I20260806 18:33:46.746883 2293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:46.747524 2293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:46.748679 2293 hybrid_clock.cc:648] HybridClock initialized: now 1786041226748660 us; error 28 us; skew 500 ppm
I20260806 18:33:46.750491 2293 webserver.cc:460] Webserver started at http://127.7.67.193:36703/ using document root <none> and password file <none>
I20260806 18:33:46.751104 2293 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:46.751170 2293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:46.751374 2293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:46.753425 2293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "172f55fe1ca04a28b681350bc46c2fb4"
format_stamp: "Formatted at 2026-08-06 18:33:46 on dist-test-slave-5bn2"
server_key: "c66cbf9cc4d34401570336dbe381d425"
server_key_iv: "164fc57db8b1bb8ea771ea9e985323b1"
server_key_version: "encryptionkey@0"
I20260806 18:33:46.753976 2293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "172f55fe1ca04a28b681350bc46c2fb4"
format_stamp: "Formatted at 2026-08-06 18:33:46 on dist-test-slave-5bn2"
server_key: "c66cbf9cc4d34401570336dbe381d425"
server_key_iv: "164fc57db8b1bb8ea771ea9e985323b1"
server_key_version: "encryptionkey@0"
I20260806 18:33:46.757717 2293 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 18:33:46.760202 2315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:46.761457 2293 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:33:46.761596 2293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "172f55fe1ca04a28b681350bc46c2fb4"
format_stamp: "Formatted at 2026-08-06 18:33:46 on dist-test-slave-5bn2"
server_key: "c66cbf9cc4d34401570336dbe381d425"
server_key_iv: "164fc57db8b1bb8ea771ea9e985323b1"
server_key_version: "encryptionkey@0"
I20260806 18:33:46.761703 2293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:46.790995 2293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:46.791756 2293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:46.791942 2293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:46.792551 2293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:46.793520 2293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:46.793570 2293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:46.793624 2293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:46.793653 2293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:46.802302 2293 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:39935
I20260806 18:33:46.802345 2475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:39935 every 8 connection(s)
I20260806 18:33:46.803236 2293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:33:46.807651 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 2293
I20260806 18:33:46.807780 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:33:46.808121 7439 external_mini_cluster.cc:1428] Setting key ec4695b6eef96e2b7d291cf1c9abfe0f
I20260806 18:33:46.810276 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:38925
--builtin_ntp_servers=127.7.67.212:35195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:33:46.813352 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.805074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:53451 (local address 127.7.67.254:38925)
0806 18:33:46.805223 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:46.805227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:46.806270 (+ 1043us) server_negotiation.cc:407] Connection header received
0806 18:33:46.807741 (+ 1471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:46.807744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:46.807805 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:46.807886 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:46.809038 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.809751 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.811322 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.811547 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.811627 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:46.812882 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:46.812886 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:46.812889 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:46.812903 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:46.812927 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:46.812938 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:46.813014 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:46.813131 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 18:33:46.813204 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:33:46.814169 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.805453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:38925 (local address 127.7.67.193:53451)
0806 18:33:46.806114 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:46.806156 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:46.807525 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:46.808109 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:46.808118 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:46.808229 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:46.808884 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:46.808905 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.809879 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.809883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:46.811210 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:46.811222 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.811694 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.811698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:46.811889 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:46.812588 (+ 699us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:33:46.812590 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:46.812645 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:33:46.812724 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:46.813014 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:46.813019 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:46.813023 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:33:46.813109 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 18:33:46.813323 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":117,"threads_started":1}
I20260806 18:33:46.815482 2476 heartbeater.cc:347] Connected to a master server at 127.7.67.254:38925
I20260806 18:33:46.815712 2476 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:46.816231 2476 heartbeater.cc:511] Master 127.7.67.254:38925 requested a full tablet report, sending...
I20260806 18:33:46.817952 2206 ts_manager.cc:194] Registered new tserver with Master: 172f55fe1ca04a28b681350bc46c2fb4 (127.7.67.193:39935)
I20260806 18:33:46.819602 2206 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.193:53451
W20260806 18:33:46.925786 2479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:46.926054 2479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:46.926090 2479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:46.930083 2479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:46.930147 2479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:46.930200 2479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:33:46.934940 2479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:35195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:38925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:46.936031 2479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:46.937409 2479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:46.947149 2488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:46.947140 2484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:46.947140 2486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:46.947412 2479 server_base.cc:1034] running on GCE node
I20260806 18:33:46.947993 2479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:46.948601 2479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:46.949759 2479 hybrid_clock.cc:648] HybridClock initialized: now 1786041226949739 us; error 31 us; skew 500 ppm
I20260806 18:33:46.951692 2479 webserver.cc:460] Webserver started at http://127.7.67.194:39507/ using document root <none> and password file <none>
I20260806 18:33:46.952273 2479 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:46.952343 2479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:46.952553 2479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:46.954668 2479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "7b1fe178fd384cb9a2f797626cfee5ff"
format_stamp: "Formatted at 2026-08-06 18:33:46 on dist-test-slave-5bn2"
server_key: "db8d1293fad36937974308bf2f92dcaf"
server_key_iv: "48f78c02b0e1562eaa0ee268fc66dbc8"
server_key_version: "encryptionkey@0"
I20260806 18:33:46.955276 2479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "7b1fe178fd384cb9a2f797626cfee5ff"
format_stamp: "Formatted at 2026-08-06 18:33:46 on dist-test-slave-5bn2"
server_key: "db8d1293fad36937974308bf2f92dcaf"
server_key_iv: "48f78c02b0e1562eaa0ee268fc66dbc8"
server_key_version: "encryptionkey@0"
I20260806 18:33:46.959000 2479 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:33:46.961436 2497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:46.962675 2479 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:33:46.962792 2479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "7b1fe178fd384cb9a2f797626cfee5ff"
format_stamp: "Formatted at 2026-08-06 18:33:46 on dist-test-slave-5bn2"
server_key: "db8d1293fad36937974308bf2f92dcaf"
server_key_iv: "48f78c02b0e1562eaa0ee268fc66dbc8"
server_key_version: "encryptionkey@0"
I20260806 18:33:46.962939 2479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:46.975844 2479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:46.976652 2479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:46.976852 2479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:46.977520 2479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:46.978581 2479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:46.978636 2479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:46.978695 2479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:46.978720 2479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:46.987835 2479 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:40295
I20260806 18:33:46.987882 2666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:40295 every 8 connection(s)
I20260806 18:33:46.988826 2479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:33:46.997912 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 2479
I20260806 18:33:46.998086 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:33:46.998201 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.990761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:46257 (local address 127.7.67.254:38925)
0806 18:33:46.990946 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:46.990951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:46.992122 (+ 1171us) server_negotiation.cc:407] Connection header received
0806 18:33:46.993538 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:46.993540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:46.993593 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:46.993658 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:46.994583 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.995179 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.996301 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.996503 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.996571 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:46.997625 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:46.997629 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:46.997631 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:46.997646 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:46.997667 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:46.997676 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:46.997732 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:46.997942 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 18:33:46.998014 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:33:46.998426 7439 external_mini_cluster.cc:1428] Setting key f1a738b9d0f9431dbd69229505b8f685
I20260806 18:33:46.999264 2672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:46.991248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:38925 (local address 127.7.67.194:46257)
0806 18:33:46.991949 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:46.992003 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:46.993323 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:46.993827 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:46.993834 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:46.993919 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:46.994466 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:46.994480 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.995304 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.995307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:46.996209 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:46.996219 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:46.996619 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:46.996622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:46.996760 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:46.997345 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:33:46.997347 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:46.997388 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:33:46.997458 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:46.997784 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:46.997795 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:46.997802 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:33:46.997941 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 18:33:46.998263 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":130,"threads_started":1}
I20260806 18:33:47.000429 2668 heartbeater.cc:347] Connected to a master server at 127.7.67.254:38925
I20260806 18:33:47.000659 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:38925
--builtin_ntp_servers=127.7.67.212:35195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:33:47.000700 2668 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:47.001329 2668 heartbeater.cc:511] Master 127.7.67.254:38925 requested a full tablet report, sending...
I20260806 18:33:47.002499 2206 ts_manager.cc:194] Registered new tserver with Master: 7b1fe178fd384cb9a2f797626cfee5ff (127.7.67.194:40295)
I20260806 18:33:47.003281 2206 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.194:46257
W20260806 18:33:47.117686 2675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:47.117978 2675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:47.118045 2675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:47.122253 2675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:47.122334 2675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:47.122413 2675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:33:47.127419 2675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:35195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:38925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:47.128615 2675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:47.130053 2675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:47.139739 2683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:47.139806 2684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:47.139780 2688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:47.140094 2675 server_base.cc:1034] running on GCE node
I20260806 18:33:47.140463 2675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:47.141086 2675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:47.142241 2675 hybrid_clock.cc:648] HybridClock initialized: now 1786041227142220 us; error 31 us; skew 500 ppm
I20260806 18:33:47.144259 2675 webserver.cc:460] Webserver started at http://127.7.67.195:36423/ using document root <none> and password file <none>
I20260806 18:33:47.144837 2675 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:47.144902 2675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:47.145123 2675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:47.147248 2675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "07323d7ee56b4999af3779d9088f812b"
format_stamp: "Formatted at 2026-08-06 18:33:47 on dist-test-slave-5bn2"
server_key: "87c140ae6124f493084f17b99f2d37fa"
server_key_iv: "b69e6c2d6f8944bd145b4b362e2b84f2"
server_key_version: "encryptionkey@0"
I20260806 18:33:47.147807 2675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "07323d7ee56b4999af3779d9088f812b"
format_stamp: "Formatted at 2026-08-06 18:33:47 on dist-test-slave-5bn2"
server_key: "87c140ae6124f493084f17b99f2d37fa"
server_key_iv: "b69e6c2d6f8944bd145b4b362e2b84f2"
server_key_version: "encryptionkey@0"
I20260806 18:33:47.151650 2675 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:33:47.154026 2699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:47.155252 2675 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260806 18:33:47.155361 2675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "07323d7ee56b4999af3779d9088f812b"
format_stamp: "Formatted at 2026-08-06 18:33:47 on dist-test-slave-5bn2"
server_key: "87c140ae6124f493084f17b99f2d37fa"
server_key_iv: "b69e6c2d6f8944bd145b4b362e2b84f2"
server_key_version: "encryptionkey@0"
I20260806 18:33:47.155459 2675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:47.177204 2675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:47.177949 2675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:47.178145 2675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:47.178751 2675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:47.179848 2675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:47.179898 2675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:47.179955 2675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:47.179992 2675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:47.188720 2675 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:44635
I20260806 18:33:47.188753 2871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:44635 every 8 connection(s)
I20260806 18:33:47.189666 2675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:33:47.198388 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:47.191583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:45415 (local address 127.7.67.254:38925)
0806 18:33:47.191725 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:47.191728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:47.192759 (+ 1031us) server_negotiation.cc:407] Connection header received
0806 18:33:47.194168 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:47.194171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:47.194220 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:47.194295 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:47.195175 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.195704 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.196750 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.196967 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.197041 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:47.198029 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:47.198032 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:47.198034 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:47.198044 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:47.198059 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:47.198065 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:47.198113 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:47.198220 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 18:33:47.198268 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:33:47.199259 2876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:47.191973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:38925 (local address 127.7.67.195:45415)
0806 18:33:47.192612 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:47.192654 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:47.193953 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:47.194443 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:47.194449 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:47.194530 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:47.195046 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:47.195060 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.195790 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.195793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:47.196652 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:47.196661 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.197080 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.197082 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:47.197214 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:47.197803 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:33:47.197805 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:47.197846 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:33:47.197909 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:47.198121 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:47.198126 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:47.198129 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:33:47.198211 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 18:33:47.198422 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":106,"threads_started":1}
I20260806 18:33:47.199878 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 2675
I20260806 18:33:47.199991 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:33:47.200280 7439 external_mini_cluster.cc:1428] Setting key adeb6a844b0edeb922653d93b5071dd0
I20260806 18:33:47.200390 2873 heartbeater.cc:347] Connected to a master server at 127.7.67.254:38925
I20260806 18:33:47.200635 2873 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:47.201236 2873 heartbeater.cc:511] Master 127.7.67.254:38925 requested a full tablet report, sending...
I20260806 18:33:47.202232 2206 ts_manager.cc:194] Registered new tserver with Master: 07323d7ee56b4999af3779d9088f812b (127.7.67.195:44635)
I20260806 18:33:47.202772 2206 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.195:45415
I20260806 18:33:47.213981 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:33:47.219183 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:47.215892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35646 (local address 127.7.67.254:38925)
0806 18:33:47.216002 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:47.216005 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:47.216150 (+ 145us) server_negotiation.cc:407] Connection header received
0806 18:33:47.216251 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:47.216253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:47.216290 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:47.216351 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:47.216583 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.217076 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.217835 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.218031 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.218097 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:47.218781 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:47.218784 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:47.218786 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:47.218798 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:47.218818 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:47.218826 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:47.218965 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:47.219034 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 18:33:47.219067 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:33:47.221892 2206 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35646:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:33:47.224929 2206 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:33:47.239058 2895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:47.233891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39008 (local address 127.7.67.195:44635)
0806 18:33:47.234375 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:47.234387 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:47.234412 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:33:47.234490 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:47.234494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:47.234693 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:47.234838 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:47.235345 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.236069 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.237618 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.238303 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.238400 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:47.238599 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:47.238750 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 18:33:47.238882 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":168,"threads_started":1}
I20260806 18:33:47.239095 2889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:47.233041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:40295 (local address 127.0.0.1:54200)
0806 18:33:47.233693 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:47.233744 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:47.233873 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:47.234868 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:47.234875 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:47.234896 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:47.235037 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:47.235044 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.236503 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.236507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:47.237607 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:47.237617 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.238537 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.238541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:47.238656 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:47.238681 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:33:47.238732 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 18:33:47.238884 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":9,"thread_start_us":90,"threads_started":1}
I20260806 18:33:47.239270 2890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:47.233381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:39935 (local address 127.0.0.1:57482)
0806 18:33:47.234503 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:47.234514 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:47.234605 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:47.235034 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:47.235037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:47.235070 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:47.235214 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:47.235220 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.236691 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.236694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:47.237901 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:47.237912 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.238916 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.238919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:47.239009 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:47.239031 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:33:47.239084 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 18:33:47.239148 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1055,"mutex_wait_us":72,"spinlock_wait_cycles":8448,"thread_start_us":106,"threads_started":1}
I20260806 18:33:47.239099 2893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:47.233076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:44635 (local address 127.0.0.1:39008)
0806 18:33:47.234226 (+ 1150us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:47.234239 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:47.234324 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:47.235038 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:47.235041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:47.235072 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:47.235206 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:47.235212 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.236182 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.236185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:47.237474 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:47.237485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.238417 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.238421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:47.238529 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:47.238605 (+ 76us) client_negotiation.cc:769] Sending connection context
0806 18:33:47.238709 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 18:33:47.238882 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1090,"mutex_wait_us":76,"spinlock_wait_cycles":3072,"thread_start_us":94,"threads_started":1}
I20260806 18:33:47.239943 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:47.233861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57482 (local address 127.7.67.193:39935)
0806 18:33:47.234185 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:47.234197 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:47.234606 (+ 409us) server_negotiation.cc:407] Connection header received
0806 18:33:47.234710 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:47.234715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:47.234905 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:47.235039 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:47.235309 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.236421 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.238026 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.238783 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.239426 (+ 643us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:47.239639 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:47.239689 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 18:33:47.239781 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":98,"threads_started":1}
I20260806 18:33:47.240128 2892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:47.233196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54200 (local address 127.7.67.194:40295)
0806 18:33:47.234390 (+ 1194us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:47.234402 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:47.234426 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:33:47.234505 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:47.234511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:47.234705 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:47.234840 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:47.235161 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.236375 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.237720 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.238395 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.239658 (+ 1263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:47.239834 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:47.239881 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 18:33:47.239979 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1093,"thread_start_us":110,"threads_started":1}
I20260806 18:33:47.242425 2565 tablet_service.cc:1467] Processing CreateTablet for tablet 666e01c7691d43bb9d3b6fcb85910d86 (DEFAULT_TABLE table=test-table [id=fd775645e5cf4c60a9599598b803b392]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:47.242491 2770 tablet_service.cc:1467] Processing CreateTablet for tablet 666e01c7691d43bb9d3b6fcb85910d86 (DEFAULT_TABLE table=test-table [id=fd775645e5cf4c60a9599598b803b392]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:47.242484 2380 tablet_service.cc:1467] Processing CreateTablet for tablet 666e01c7691d43bb9d3b6fcb85910d86 (DEFAULT_TABLE table=test-table [id=fd775645e5cf4c60a9599598b803b392]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:47.243435 2565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666e01c7691d43bb9d3b6fcb85910d86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:47.243534 2380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666e01c7691d43bb9d3b6fcb85910d86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:47.243534 2770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666e01c7691d43bb9d3b6fcb85910d86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:47.249552 2898 tablet_bootstrap.cc:492] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff: Bootstrap starting.
I20260806 18:33:47.251055 2899 tablet_bootstrap.cc:492] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b: Bootstrap starting.
I20260806 18:33:47.251353 2900 tablet_bootstrap.cc:492] T 666e01c7691d43bb9d3b6fcb85910d86 P 172f55fe1ca04a28b681350bc46c2fb4: Bootstrap starting.
I20260806 18:33:47.251958 2898 tablet_bootstrap.cc:654] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:47.252825 2898 log.cc:824] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:47.253186 2899 tablet_bootstrap.cc:654] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:47.253186 2900 tablet_bootstrap.cc:654] T 666e01c7691d43bb9d3b6fcb85910d86 P 172f55fe1ca04a28b681350bc46c2fb4: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:47.254021 2899 log.cc:824] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:47.254021 2900 log.cc:824] T 666e01c7691d43bb9d3b6fcb85910d86 P 172f55fe1ca04a28b681350bc46c2fb4: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:47.254694 2898 tablet_bootstrap.cc:492] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff: No bootstrap required, opened a new log
I20260806 18:33:47.254951 2898 ts_tablet_manager.cc:1397] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:33:47.256035 2899 tablet_bootstrap.cc:492] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b: No bootstrap required, opened a new log
I20260806 18:33:47.256229 2899 ts_tablet_manager.cc:1397] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:33:47.256423 2900 tablet_bootstrap.cc:492] T 666e01c7691d43bb9d3b6fcb85910d86 P 172f55fe1ca04a28b681350bc46c2fb4: No bootstrap required, opened a new log
I20260806 18:33:47.256610 2900 ts_tablet_manager.cc:1397] T 666e01c7691d43bb9d3b6fcb85910d86 P 172f55fe1ca04a28b681350bc46c2fb4: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:33:47.258589 2898 raft_consensus.cc:348] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b1fe178fd384cb9a2f797626cfee5ff" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 40295 } } peers { permanent_uuid: "07323d7ee56b4999af3779d9088f812b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44635 } } peers { permanent_uuid: "172f55fe1ca04a28b681350bc46c2fb4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39935 } }
I20260806 18:33:47.258811 2898 raft_consensus.cc:374] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:47.258881 2898 raft_consensus.cc:729] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b1fe178fd384cb9a2f797626cfee5ff, State: Initialized, Role: FOLLOWER
I20260806 18:33:47.259342 2898 consensus_queue.cc:260] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b1fe178fd384cb9a2f797626cfee5ff" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 40295 } } peers { permanent_uuid: "07323d7ee56b4999af3779d9088f812b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44635 } } peers { permanent_uuid: "172f55fe1ca04a28b681350bc46c2fb4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39935 } }
I20260806 18:33:47.260167 2668 heartbeater.cc:503] Master 127.7.67.254:38925 was elected leader, sending a full tablet report...
I20260806 18:33:47.260366 2898 ts_tablet_manager.cc:1428] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:33:47.260682 2900 raft_consensus.cc:348] T 666e01c7691d43bb9d3b6fcb85910d86 P 172f55fe1ca04a28b681350bc46c2fb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b1fe178fd384cb9a2f797626cfee5ff" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 40295 } } peers { permanent_uuid: "07323d7ee56b4999af3779d9088f812b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44635 } } peers { permanent_uuid: "172f55fe1ca04a28b681350bc46c2fb4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39935 } }
I20260806 18:33:47.260936 2900 raft_consensus.cc:374] T 666e01c7691d43bb9d3b6fcb85910d86 P 172f55fe1ca04a28b681350bc46c2fb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:47.261004 2900 raft_consensus.cc:729] T 666e01c7691d43bb9d3b6fcb85910d86 P 172f55fe1ca04a28b681350bc46c2fb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 172f55fe1ca04a28b681350bc46c2fb4, State: Initialized, Role: FOLLOWER
I20260806 18:33:47.260938 2899 raft_consensus.cc:348] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b1fe178fd384cb9a2f797626cfee5ff" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 40295 } } peers { permanent_uuid: "07323d7ee56b4999af3779d9088f812b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44635 } } peers { permanent_uuid: "172f55fe1ca04a28b681350bc46c2fb4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39935 } }
I20260806 18:33:47.261190 2899 raft_consensus.cc:374] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:47.261245 2899 raft_consensus.cc:729] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07323d7ee56b4999af3779d9088f812b, State: Initialized, Role: FOLLOWER
I20260806 18:33:47.261533 2900 consensus_queue.cc:260] T 666e01c7691d43bb9d3b6fcb85910d86 P 172f55fe1ca04a28b681350bc46c2fb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b1fe178fd384cb9a2f797626cfee5ff" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 40295 } } peers { permanent_uuid: "07323d7ee56b4999af3779d9088f812b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44635 } } peers { permanent_uuid: "172f55fe1ca04a28b681350bc46c2fb4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39935 } }
I20260806 18:33:47.261624 2899 consensus_queue.cc:260] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b1fe178fd384cb9a2f797626cfee5ff" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 40295 } } peers { permanent_uuid: "07323d7ee56b4999af3779d9088f812b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44635 } } peers { permanent_uuid: "172f55fe1ca04a28b681350bc46c2fb4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39935 } }
I20260806 18:33:47.262161 2476 heartbeater.cc:503] Master 127.7.67.254:38925 was elected leader, sending a full tablet report...
I20260806 18:33:47.262490 2899 ts_tablet_manager.cc:1428] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260806 18:33:47.262827 2873 heartbeater.cc:503] Master 127.7.67.254:38925 was elected leader, sending a full tablet report...
I20260806 18:33:47.263337 2900 ts_tablet_manager.cc:1428] T 666e01c7691d43bb9d3b6fcb85910d86 P 172f55fe1ca04a28b681350bc46c2fb4: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260806 18:33:47.304471 2477 tablet.cc:2369] T 666e01c7691d43bb9d3b6fcb85910d86 P 172f55fe1ca04a28b681350bc46c2fb4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:33:47.417315 2907 raft_consensus.cc:482] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:33:47.417562 2907 raft_consensus.cc:504] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b1fe178fd384cb9a2f797626cfee5ff" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 40295 } } peers { permanent_uuid: "07323d7ee56b4999af3779d9088f812b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44635 } } peers { permanent_uuid: "172f55fe1ca04a28b681350bc46c2fb4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39935 } }
I20260806 18:33:47.418751 2907 leader_election.cc:288] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07323d7ee56b4999af3779d9088f812b (127.7.67.195:44635), 172f55fe1ca04a28b681350bc46c2fb4 (127.7.67.193:39935)
I20260806 18:33:47.422592 2672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:47.419063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:39935 (local address 127.7.67.194:48517)
0806 18:33:47.419282 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:47.419296 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:47.419413 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:47.419879 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:47.419882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:47.419897 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:47.419993 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:47.419998 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.420687 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.420691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:47.421510 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:47.421519 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.422125 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.422129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:47.422268 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:47.422290 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:33:47.422369 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 18:33:47.422438 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":60}
I20260806 18:33:47.422587 2895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:47.419114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:50031 (local address 127.7.67.195:44635)
0806 18:33:47.419250 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:47.419254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:47.419571 (+ 317us) server_negotiation.cc:407] Connection header received
0806 18:33:47.419660 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:47.419663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:47.419712 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:47.419778 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:47.420033 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.420534 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.421641 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.422040 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.422143 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:47.422191 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:47.422395 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 18:33:47.422464 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:33:47.422600 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:47.419159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:48517 (local address 127.7.67.193:39935)
0806 18:33:47.419269 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:47.419274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:47.419372 (+ 98us) server_negotiation.cc:407] Connection header received
0806 18:33:47.419517 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:47.419519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:47.419571 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:47.419642 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:47.420100 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.420588 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.421625 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.422019 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.422125 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:47.422177 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:47.422409 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 18:33:47.422479 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 18:33:47.422794 2911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:47.419114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:44635 (local address 127.7.67.194:50031)
0806 18:33:47.419477 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:47.419488 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:47.419583 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:47.419796 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:47.419798 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:47.419809 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:47.419919 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:47.419923 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.420637 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.420641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:47.421518 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:47.421527 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.422143 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.422147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:47.422291 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:47.422309 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:33:47.422373 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 18:33:47.422449 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":54,"thread_start_us":148,"threads_started":1}
I20260806 18:33:47.423499 2408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666e01c7691d43bb9d3b6fcb85910d86" candidate_uuid: "7b1fe178fd384cb9a2f797626cfee5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "172f55fe1ca04a28b681350bc46c2fb4" is_pre_election: true
I20260806 18:33:47.423547 2798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666e01c7691d43bb9d3b6fcb85910d86" candidate_uuid: "7b1fe178fd384cb9a2f797626cfee5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07323d7ee56b4999af3779d9088f812b" is_pre_election: true
I20260806 18:33:47.423836 2408 raft_consensus.cc:2454] T 666e01c7691d43bb9d3b6fcb85910d86 P 172f55fe1ca04a28b681350bc46c2fb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b1fe178fd384cb9a2f797626cfee5ff in term 0.
I20260806 18:33:47.423838 2798 raft_consensus.cc:2454] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b1fe178fd384cb9a2f797626cfee5ff in term 0.
I20260806 18:33:47.424366 2498 leader_election.cc:302] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 07323d7ee56b4999af3779d9088f812b, 172f55fe1ca04a28b681350bc46c2fb4, 7b1fe178fd384cb9a2f797626cfee5ff; no voters:
I20260806 18:33:47.424638 2907 raft_consensus.cc:2781] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:33:47.424732 2907 raft_consensus.cc:482] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:33:47.424782 2907 raft_consensus.cc:3037] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:47.425833 2907 raft_consensus.cc:504] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b1fe178fd384cb9a2f797626cfee5ff" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 40295 } } peers { permanent_uuid: "07323d7ee56b4999af3779d9088f812b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44635 } } peers { permanent_uuid: "172f55fe1ca04a28b681350bc46c2fb4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39935 } }
I20260806 18:33:47.426186 2907 leader_election.cc:288] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [CANDIDATE]: Term 1 election: Requested vote from peers 07323d7ee56b4999af3779d9088f812b (127.7.67.195:44635), 172f55fe1ca04a28b681350bc46c2fb4 (127.7.67.193:39935)
I20260806 18:33:47.426586 2798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666e01c7691d43bb9d3b6fcb85910d86" candidate_uuid: "7b1fe178fd384cb9a2f797626cfee5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07323d7ee56b4999af3779d9088f812b"
I20260806 18:33:47.426628 2408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666e01c7691d43bb9d3b6fcb85910d86" candidate_uuid: "7b1fe178fd384cb9a2f797626cfee5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "172f55fe1ca04a28b681350bc46c2fb4"
I20260806 18:33:47.426705 2798 raft_consensus.cc:3037] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:47.426738 2408 raft_consensus.cc:3037] T 666e01c7691d43bb9d3b6fcb85910d86 P 172f55fe1ca04a28b681350bc46c2fb4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:47.427752 2798 raft_consensus.cc:2454] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b1fe178fd384cb9a2f797626cfee5ff in term 1.
I20260806 18:33:47.427825 2408 raft_consensus.cc:2454] T 666e01c7691d43bb9d3b6fcb85910d86 P 172f55fe1ca04a28b681350bc46c2fb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b1fe178fd384cb9a2f797626cfee5ff in term 1.
I20260806 18:33:47.428076 2498 leader_election.cc:302] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07323d7ee56b4999af3779d9088f812b, 7b1fe178fd384cb9a2f797626cfee5ff; no voters:
I20260806 18:33:47.428308 2907 raft_consensus.cc:2781] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:33:47.428525 2907 raft_consensus.cc:686] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [term 1 LEADER]: Becoming Leader. State: Replica: 7b1fe178fd384cb9a2f797626cfee5ff, State: Running, Role: LEADER
I20260806 18:33:47.428835 2907 consensus_queue.cc:237] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b1fe178fd384cb9a2f797626cfee5ff" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 40295 } } peers { permanent_uuid: "07323d7ee56b4999af3779d9088f812b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44635 } } peers { permanent_uuid: "172f55fe1ca04a28b681350bc46c2fb4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39935 } }
I20260806 18:33:47.432039 2206 catalog_manager.cc:5495] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b1fe178fd384cb9a2f797626cfee5ff (127.7.67.194). New cstate: current_term: 1 leader_uuid: "7b1fe178fd384cb9a2f797626cfee5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b1fe178fd384cb9a2f797626cfee5ff" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 40295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07323d7ee56b4999af3779d9088f812b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "172f55fe1ca04a28b681350bc46c2fb4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39935 } health_report { overall_health: UNKNOWN } } }
W20260806 18:33:47.440614 2874 tablet.cc:2369] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:33:47.478513 2892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:47.474970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54214 (local address 127.7.67.194:40295)
0806 18:33:47.475125 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:47.475130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:47.475145 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:33:47.475251 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:47.475254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:47.475313 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:47.475405 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:33:47.475664 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.476215 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.477044 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:47.477262 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:47.477348 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:47.477658 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:33:47.478083 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:33:47.478214 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:47.478319 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 18:33:47.478355 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:33:47.484447 2798 raft_consensus.cc:1261] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b [term 1 FOLLOWER]: Refusing update from remote peer 7b1fe178fd384cb9a2f797626cfee5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:33:47.484447 2408 raft_consensus.cc:1261] T 666e01c7691d43bb9d3b6fcb85910d86 P 172f55fe1ca04a28b681350bc46c2fb4 [term 1 FOLLOWER]: Refusing update from remote peer 7b1fe178fd384cb9a2f797626cfee5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:33:47.485162 2907 consensus_queue.cc:1035] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "07323d7ee56b4999af3779d9088f812b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 44635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:33:47.485351 2912 consensus_queue.cc:1035] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "172f55fe1ca04a28b681350bc46c2fb4" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 39935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:33:47.489956 2670 tablet.cc:2369] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:33:47.493366 2918 mvcc.cc:205] Tried to move back new op lower bound from 7315624867768442880 to 7315624867551072256. Current Snapshot: MvccSnapshot[applied={T|T < 7315624867768442880}]
I20260806 18:33:47.493643 2917 mvcc.cc:205] Tried to move back new op lower bound from 7315624867768442880 to 7315624867551072256. Current Snapshot: MvccSnapshot[applied={T|T < 7315624867768442880}]
I20260806 18:33:47.501587 2206 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:33:47.501777 2206 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:33:47.504268 2206 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9fd47a75f9a4bb8b6e4f03285177460: Sending DeleteTablet for 3 replicas of tablet 666e01c7691d43bb9d3b6fcb85910d86
I20260806 18:33:47.505025 2770 tablet_service.cc:1514] Processing DeleteTablet for tablet 666e01c7691d43bb9d3b6fcb85910d86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:47 UTC) from {username='slave'} at 127.0.0.1:39008
I20260806 18:33:47.505050 2380 tablet_service.cc:1514] Processing DeleteTablet for tablet 666e01c7691d43bb9d3b6fcb85910d86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:47 UTC) from {username='slave'} at 127.0.0.1:57482
I20260806 18:33:47.505517 2930 tablet_replica.cc:330] T 666e01c7691d43bb9d3b6fcb85910d86 P 172f55fe1ca04a28b681350bc46c2fb4: stopping tablet replica
I20260806 18:33:47.505514 2929 tablet_replica.cc:330] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b: stopping tablet replica
I20260806 18:33:47.505846 2929 raft_consensus.cc:2229] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:33:47.505846 2930 raft_consensus.cc:2229] T 666e01c7691d43bb9d3b6fcb85910d86 P 172f55fe1ca04a28b681350bc46c2fb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:33:47.506182 2930 raft_consensus.cc:2258] T 666e01c7691d43bb9d3b6fcb85910d86 P 172f55fe1ca04a28b681350bc46c2fb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:33:47.506327 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 2293
I20260806 18:33:47.506347 2565 tablet_service.cc:1514] Processing DeleteTablet for tablet 666e01c7691d43bb9d3b6fcb85910d86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:47 UTC) from {username='slave'} at 127.0.0.1:54200
I20260806 18:33:47.510093 2935 tablet_replica.cc:330] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff: stopping tablet replica
I20260806 18:33:47.510404 2935 raft_consensus.cc:2229] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:33:47.510735 2935 raft_consensus.cc:2258] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:33:47.512073 2935 ts_tablet_manager.cc:1905] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:33:47.514226 2935 ts_tablet_manager.cc:1918] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:33:47.514317 2935 log.cc:1196] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/666e01c7691d43bb9d3b6fcb85910d86
I20260806 18:33:47.514631 2935 ts_tablet_manager.cc:1939] T 666e01c7691d43bb9d3b6fcb85910d86 P 7b1fe178fd384cb9a2f797626cfee5ff: Deleting consensus metadata
I20260806 18:33:47.514881 2929 raft_consensus.cc:2258] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:33:47.516047 2186 catalog_manager.cc:4841] TS 7b1fe178fd384cb9a2f797626cfee5ff (127.7.67.194:40295): tablet 666e01c7691d43bb9d3b6fcb85910d86 (table test-table [id=fd775645e5cf4c60a9599598b803b392]) successfully deleted
I20260806 18:33:47.516570 2929 ts_tablet_manager.cc:1905] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 18:33:47.518388 2187 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.193:39935 (error 108)
I20260806 18:33:47.518772 2929 ts_tablet_manager.cc:1918] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:33:47.518899 2929 log.cc:1196] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/666e01c7691d43bb9d3b6fcb85910d86
I20260806 18:33:47.519213 2929 ts_tablet_manager.cc:1939] T 666e01c7691d43bb9d3b6fcb85910d86 P 07323d7ee56b4999af3779d9088f812b: Deleting consensus metadata
I20260806 18:33:47.519284 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 2479
I20260806 18:33:47.521528 2184 catalog_manager.cc:4841] TS 07323d7ee56b4999af3779d9088f812b (127.7.67.195:44635): tablet 666e01c7691d43bb9d3b6fcb85910d86 (table test-table [id=fd775645e5cf4c60a9599598b803b392]) successfully deleted
I20260806 18:33:47.522742 2893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:47.522377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:39935 (local address 127.0.0.1:57490)
0806 18:33:47.522521 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:47.522615 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:39935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 18:33:47.523979 2187 catalog_manager.cc:4584] TS 172f55fe1ca04a28b681350bc46c2fb4 (127.7.67.193:39935): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 666e01c7691d43bb9d3b6fcb85910d86: Network error: Client connection negotiation failed: client connection to 127.7.67.193:39935: connect: Connection refused (error 111)
I20260806 18:33:47.529704 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 2675
I20260806 18:33:47.541967 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 2160
2026-08-06T18:33:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1212 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:33:47 dist-test-slave-5bn2 krb5kdc[2940](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:33:47 dist-test-slave-5bn2 krb5kdc[2940](info): set up 2 sockets
Aug 06 18:33:47 dist-test-slave-5bn2 krb5kdc[2940](info): commencing operation
krb5kdc: starting...
W20260806 18:33:48.662765 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:33:48 dist-test-slave-5bn2 krb5kdc[2940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:33:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:33:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:33:48.785812 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:36907
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:44217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:36907
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:33:48.904354 2961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:48.904688 2961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:48.904737 2961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:48.909780 2961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:33:48.909857 2961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:48.909885 2961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:48.909909 2961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:33:48.909938 2961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:33:48.917024 2961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:44217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:36907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:36907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:48.918840 2961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:48.920962 2961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:48.930601 2970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:48.930595 2969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:48.931185 2973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:48.931630 2961 server_base.cc:1034] running on GCE node
I20260806 18:33:48.932221 2961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:48.933378 2961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:48.934556 2961 hybrid_clock.cc:648] HybridClock initialized: now 1786041228934528 us; error 42 us; skew 500 ppm
Aug 06 18:33:48 dist-test-slave-5bn2 krb5kdc[2940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041228, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:48.943365 2961 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:33:48.944316 2961 webserver.cc:460] Webserver started at http://127.7.67.254:41921/ using document root <none> and password file <none>
I20260806 18:33:48.944887 2961 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:48.944952 2961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:48.945174 2961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:48.947388 2961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "1c299b9456784748859a2087bb76b429"
format_stamp: "Formatted at 2026-08-06 18:33:48 on dist-test-slave-5bn2"
server_key: "101fa4f627ff66d62d39e5498f9939ef"
server_key_iv: "14c93f9f6c12270f556018bf85de7969"
server_key_version: "encryptionkey@0"
I20260806 18:33:48.947937 2961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "1c299b9456784748859a2087bb76b429"
format_stamp: "Formatted at 2026-08-06 18:33:48 on dist-test-slave-5bn2"
server_key: "101fa4f627ff66d62d39e5498f9939ef"
server_key_iv: "14c93f9f6c12270f556018bf85de7969"
server_key_version: "encryptionkey@0"
I20260806 18:33:48.951709 2961 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:33:48.954133 2984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:48.955528 2961 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:33:48.955667 2961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "1c299b9456784748859a2087bb76b429"
format_stamp: "Formatted at 2026-08-06 18:33:48 on dist-test-slave-5bn2"
server_key: "101fa4f627ff66d62d39e5498f9939ef"
server_key_iv: "14c93f9f6c12270f556018bf85de7969"
server_key_version: "encryptionkey@0"
I20260806 18:33:48.955775 2961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:48.974047 2961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:48.977078 2961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:48.977276 2961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:48.984427 2961 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:36907
I20260806 18:33:48.984472 3058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:36907 every 8 connection(s)
I20260806 18:33:48.985448 2961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:33:48.988375 3059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:48.994203 3059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429: Bootstrap starting.
I20260806 18:33:48.995391 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 2961
I20260806 18:33:48.995529 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:33:48.995796 7439 external_mini_cluster.cc:1428] Setting key 3a358edc0dd54cfc0713cf63a5b313c5
I20260806 18:33:48.996980 3059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:48.997910 3059 log.cc:824] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:49.000186 3059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429: No bootstrap required, opened a new log
I20260806 18:33:49.004451 3059 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c299b9456784748859a2087bb76b429" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 36907 } }
I20260806 18:33:49.004758 3059 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:49.004823 3059 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c299b9456784748859a2087bb76b429, State: Initialized, Role: FOLLOWER
Aug 06 18:33:49 dist-test-slave-5bn2 krb5kdc[2940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:33:49.005313 3059 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c299b9456784748859a2087bb76b429" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 36907 } }
I20260806 18:33:49.005462 3059 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:33:49.005529 3059 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:33:49.005625 3059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:49.006716 3059 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c299b9456784748859a2087bb76b429" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 36907 } }
I20260806 18:33:49.007092 3059 leader_election.cc:302] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c299b9456784748859a2087bb76b429; no voters:
I20260806 18:33:49.007445 3059 leader_election.cc:288] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:33:49.007552 3065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:33:49.007721 3065 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429 [term 1 LEADER]: Becoming Leader. State: Replica: 1c299b9456784748859a2087bb76b429, State: Running, Role: LEADER
I20260806 18:33:49.008023 3065 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c299b9456784748859a2087bb76b429" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 36907 } }
I20260806 18:33:49.008834 3059 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:33:49.009250 3065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c299b9456784748859a2087bb76b429. Latest consensus state: current_term: 1 leader_uuid: "1c299b9456784748859a2087bb76b429" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c299b9456784748859a2087bb76b429" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 36907 } } }
I20260806 18:33:49.009366 3065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429 [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:49.009672 3068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c299b9456784748859a2087bb76b429" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c299b9456784748859a2087bb76b429" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 36907 } } }
I20260806 18:33:49.009768 3068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429 [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:49.011358 3062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:48.996905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47284 (local address 127.7.67.254:36907)
0806 18:33:48.997398 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:48.997419 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:48.997463 (+ 44us) server_negotiation.cc:407] Connection header received
0806 18:33:48.998300 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:48.998325 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:48.998686 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:48.999073 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:48.999729 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.001199 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.002025 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.002752 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.002956 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:49.005670 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:49.005677 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:49.005697 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:49.005731 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:49.007467 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:49.008001 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:49.008008 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:49.008014 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:49.008154 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:49.008414 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:49.008417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:49.008419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:49.008889 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:49.009081 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:49.009988 (+ 907us) server_negotiation.cc:299] Negotiation successful
0806 18:33:49.010319 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":135,"threads_started":1}
W20260806 18:33:49.013617 3085 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:33:49.013693 3085 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:33:49.013772 3071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:33:49.014515 3071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:33:49.020351 3071 catalog_manager.cc:1347] Generated new cluster ID: 2b1171ee30084eab976052ba67e34d29
I20260806 18:33:49.020440 3071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:33:49.042068 3071 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:33:49.043120 3071 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:33:49.052477 3071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429: Generated new TSK 0
I20260806 18:33:49.053340 3071 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
Entry for principal kudu/127.7.67.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab.
I20260806 18:33:49.089385 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:36907
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:44217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:33:49.202220 3095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:49.202483 3095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:49.202524 3095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:49.206506 3095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:49.206568 3095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:49.206624 3095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:33:49.211968 3095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:44217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.193.keytab
--principal=kudu/127.7.67.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:49.212996 3095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:49.213995 3095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:49.223559 3106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:49.223548 3102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:49.223554 3104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:49.223887 3095 server_base.cc:1034] running on GCE node
I20260806 18:33:49.224272 3095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:49.224936 3095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:49.226104 3095 hybrid_clock.cc:648] HybridClock initialized: now 1786041229226082 us; error 32 us; skew 500 ppm
Aug 06 18:33:49 dist-test-slave-5bn2 krb5kdc[2940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041229, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:49.228801 3095 init.cc:377] Logged in from keytab as kudu/127.7.67.193@KRBTEST.COM (short username kudu)
I20260806 18:33:49.229655 3095 webserver.cc:460] Webserver started at http://127.7.67.193:36263/ using document root <none> and password file <none>
I20260806 18:33:49.230218 3095 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:49.230288 3095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:49.230501 3095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:49.232578 3095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "724ac09266c54619b8d55a6815c27a8c"
format_stamp: "Formatted at 2026-08-06 18:33:49 on dist-test-slave-5bn2"
server_key: "af975a6fad0a66522b88b69f84f059af"
server_key_iv: "4e55be721fa0a3b488d9d9e2f85f7a58"
server_key_version: "encryptionkey@0"
I20260806 18:33:49.233122 3095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "724ac09266c54619b8d55a6815c27a8c"
format_stamp: "Formatted at 2026-08-06 18:33:49 on dist-test-slave-5bn2"
server_key: "af975a6fad0a66522b88b69f84f059af"
server_key_iv: "4e55be721fa0a3b488d9d9e2f85f7a58"
server_key_version: "encryptionkey@0"
I20260806 18:33:49.236770 3095 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:33:49.239123 3119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:49.240302 3095 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:33:49.240403 3095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "724ac09266c54619b8d55a6815c27a8c"
format_stamp: "Formatted at 2026-08-06 18:33:49 on dist-test-slave-5bn2"
server_key: "af975a6fad0a66522b88b69f84f059af"
server_key_iv: "4e55be721fa0a3b488d9d9e2f85f7a58"
server_key_version: "encryptionkey@0"
I20260806 18:33:49.240520 3095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:49.273277 3095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:49.276302 3095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:49.276500 3095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:49.277122 3095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:49.277985 3095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:49.278033 3095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:49.278079 3095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:49.278108 3095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:49.286568 3095 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:40699
I20260806 18:33:49.286618 3281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:40699 every 8 connection(s)
I20260806 18:33:49.287598 3095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:33:49.296623 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 3095
I20260806 18:33:49.296753 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:33:49.297051 7439 external_mini_cluster.cc:1428] Setting key 85bd704587204c7801a29cb5aeda7385
Aug 06 18:33:49 dist-test-slave-5bn2 krb5kdc[2940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041229, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.193@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:33:49.301365 3062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:49.289382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:52147 (local address 127.7.67.254:36907)
0806 18:33:49.289530 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:49.289534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:49.290384 (+ 850us) server_negotiation.cc:407] Connection header received
0806 18:33:49.291789 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:49.291793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:49.291852 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:49.291929 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:49.293189 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.293734 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.294900 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.295140 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.295240 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:49.297727 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:49.297730 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:49.297733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:49.297759 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:49.299207 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:49.299792 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:49.299794 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:49.299795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:49.299864 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:49.300207 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:49.300209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:49.300210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:49.300374 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:49.300441 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:49.301070 (+ 629us) server_negotiation.cc:299] Negotiation successful
0806 18:33:49.301199 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:33:49.302171 3285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:49.289755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:36907 (local address 127.7.67.193:52147)
0806 18:33:49.290227 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:49.290274 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:49.291578 (+ 1304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:49.292127 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:49.292136 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:49.292534 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:49.293025 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:49.293048 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.293860 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.293863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:49.294773 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:49.294782 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.295297 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.295301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:49.295466 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:49.296047 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:33:49.296064 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:49.297539 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:49.299322 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:49.299328 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:49.299343 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:49.299706 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:49.299934 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:49.299936 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:49.299938 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:49.300126 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:49.300459 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:49.300465 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:49.300813 (+ 348us) client_negotiation.cc:769] Sending connection context
0806 18:33:49.301003 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 18:33:49.301280 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":123,"threads_started":1}
I20260806 18:33:49.303637 3283 heartbeater.cc:347] Connected to a master server at 127.7.67.254:36907
I20260806 18:33:49.303885 3283 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:49.304401 3283 heartbeater.cc:511] Master 127.7.67.254:36907 requested a full tablet report, sending...
I20260806 18:33:49.305946 3012 ts_manager.cc:194] Registered new tserver with Master: 724ac09266c54619b8d55a6815c27a8c (127.7.67.193:40699)
I20260806 18:33:49.307119 3012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.193@KRBTEST.COM'} at 127.7.67.193:52147
WARNING: no policy specified for kudu/127.7.67.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
Entry for principal kudu/127.7.67.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab.
I20260806 18:33:49.325240 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:36907
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:44217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:33:49.438323 3289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:49.438599 3289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:49.438659 3289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:49.442667 3289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:49.442732 3289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:49.442811 3289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:33:49.448149 3289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:44217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.194.keytab
--principal=kudu/127.7.67.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:49.449235 3289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:49.450425 3289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:49.460449 3298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:49.460403 3300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:49.460482 3295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:49.460708 3289 server_base.cc:1034] running on GCE node
I20260806 18:33:49.461289 3289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:49.461958 3289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:49.463110 3289 hybrid_clock.cc:648] HybridClock initialized: now 1786041229463070 us; error 56 us; skew 500 ppm
Aug 06 18:33:49 dist-test-slave-5bn2 krb5kdc[2940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041229, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:49.465914 3289 init.cc:377] Logged in from keytab as kudu/127.7.67.194@KRBTEST.COM (short username kudu)
I20260806 18:33:49.466890 3289 webserver.cc:460] Webserver started at http://127.7.67.194:37563/ using document root <none> and password file <none>
I20260806 18:33:49.467501 3289 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:49.467588 3289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:49.467813 3289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:49.469972 3289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "0d6ff63ef5124852a3c51079547e64db"
format_stamp: "Formatted at 2026-08-06 18:33:49 on dist-test-slave-5bn2"
server_key: "cabf95e0a824dea9d78de4b19c049e22"
server_key_iv: "252ea0a4bed8363797e201acc4f49bf0"
server_key_version: "encryptionkey@0"
I20260806 18:33:49.470583 3289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "0d6ff63ef5124852a3c51079547e64db"
format_stamp: "Formatted at 2026-08-06 18:33:49 on dist-test-slave-5bn2"
server_key: "cabf95e0a824dea9d78de4b19c049e22"
server_key_iv: "252ea0a4bed8363797e201acc4f49bf0"
server_key_version: "encryptionkey@0"
I20260806 18:33:49.474423 3289 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 18:33:49.477002 3310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:49.478224 3289 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:33:49.478354 3289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "0d6ff63ef5124852a3c51079547e64db"
format_stamp: "Formatted at 2026-08-06 18:33:49 on dist-test-slave-5bn2"
server_key: "cabf95e0a824dea9d78de4b19c049e22"
server_key_iv: "252ea0a4bed8363797e201acc4f49bf0"
server_key_version: "encryptionkey@0"
I20260806 18:33:49.478459 3289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:49.494140 3289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:49.497269 3289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:49.497463 3289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:49.498090 3289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:49.499037 3289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:49.499092 3289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:49.499150 3289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:49.499174 3289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:49.507692 3289 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:42703
I20260806 18:33:49.507726 3475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:42703 every 8 connection(s)
I20260806 18:33:49.508641 3289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:33:49.512355 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 3289
I20260806 18:33:49.512460 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:33:49.512722 7439 external_mini_cluster.cc:1428] Setting key e095bfca820ef483fda7ce9bb62eb408
Aug 06 18:33:49 dist-test-slave-5bn2 krb5kdc[2940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041229, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.194@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.195@KRBTEST.COM; defaulting to no policy
I20260806 18:33:49.523017 3062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:49.510704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:52671 (local address 127.7.67.254:36907)
0806 18:33:49.510839 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:49.510843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:49.511833 (+ 990us) server_negotiation.cc:407] Connection header received
0806 18:33:49.513266 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:49.513270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:49.513328 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:49.513402 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:49.514628 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.515195 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.516532 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.516763 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.516844 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:49.519595 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:49.519598 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:49.519600 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:49.519628 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:49.520953 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:49.521459 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:49.521461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:49.521463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:49.521544 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:49.521869 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:49.521872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:49.521874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:49.522066 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:49.522155 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:49.522679 (+ 524us) server_negotiation.cc:299] Negotiation successful
0806 18:33:49.522813 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:33:49.523929 3478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:49.511173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:36907 (local address 127.7.67.194:52671)
0806 18:33:49.511672 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:49.511720 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:49.513040 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:49.513592 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:49.513599 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:49.513968 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:49.514476 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:49.514490 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.515319 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.515322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:49.516382 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:49.516394 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.516925 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.516928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:49.517143 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:49.517831 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:33:49.517849 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:49.519414 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:49.521070 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:49.521077 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:49.521093 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:49.521360 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:49.521630 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:49.521634 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:49.521636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:49.521781 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:49.522170 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:49.522177 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:49.522442 (+ 265us) client_negotiation.cc:769] Sending connection context
0806 18:33:49.522619 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 18:33:49.523007 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":127,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.195@KRBTEST.COM" created.
I20260806 18:33:49.525084 3476 heartbeater.cc:347] Connected to a master server at 127.7.67.254:36907
I20260806 18:33:49.525381 3476 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:49.525925 3476 heartbeater.cc:511] Master 127.7.67.254:36907 requested a full tablet report, sending...
I20260806 18:33:49.527112 3012 ts_manager.cc:194] Registered new tserver with Master: 0d6ff63ef5124852a3c51079547e64db (127.7.67.194:42703)
I20260806 18:33:49.527882 3012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.194@KRBTEST.COM'} at 127.7.67.194:52671
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
Entry for principal kudu/127.7.67.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab.
I20260806 18:33:49.541607 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:36907
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:44217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:33:49.658129 3482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:49.658381 3482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:49.658423 3482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:49.662549 3482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:49.662621 3482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:49.662676 3482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:33:49.668005 3482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:44217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.195.keytab
--principal=kudu/127.7.67.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:49.669034 3482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:49.670042 3482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:49.679774 3492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:49.679785 3491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:49.679971 3496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:49.680053 3482 server_base.cc:1034] running on GCE node
I20260806 18:33:49.680480 3482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:49.681123 3482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:49.682283 3482 hybrid_clock.cc:648] HybridClock initialized: now 1786041229682255 us; error 38 us; skew 500 ppm
Aug 06 18:33:49 dist-test-slave-5bn2 krb5kdc[2940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041229, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:49.685101 3482 init.cc:377] Logged in from keytab as kudu/127.7.67.195@KRBTEST.COM (short username kudu)
I20260806 18:33:49.686012 3482 webserver.cc:460] Webserver started at http://127.7.67.195:36411/ using document root <none> and password file <none>
I20260806 18:33:49.686582 3482 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:49.686646 3482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:49.686839 3482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:49.689030 3482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "4618d8673e43475fbbd726441c431e6b"
format_stamp: "Formatted at 2026-08-06 18:33:49 on dist-test-slave-5bn2"
server_key: "e0b5ef93a4238d7547db1601bf49d22f"
server_key_iv: "6702873b4161f5f87ab67730a29f37f6"
server_key_version: "encryptionkey@0"
I20260806 18:33:49.689599 3482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "4618d8673e43475fbbd726441c431e6b"
format_stamp: "Formatted at 2026-08-06 18:33:49 on dist-test-slave-5bn2"
server_key: "e0b5ef93a4238d7547db1601bf49d22f"
server_key_iv: "6702873b4161f5f87ab67730a29f37f6"
server_key_version: "encryptionkey@0"
I20260806 18:33:49.693455 3482 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 18:33:49.695982 3506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:49.697225 3482 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:33:49.697345 3482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "4618d8673e43475fbbd726441c431e6b"
format_stamp: "Formatted at 2026-08-06 18:33:49 on dist-test-slave-5bn2"
server_key: "e0b5ef93a4238d7547db1601bf49d22f"
server_key_iv: "6702873b4161f5f87ab67730a29f37f6"
server_key_version: "encryptionkey@0"
I20260806 18:33:49.697474 3482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:49.709944 3482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:49.713024 3482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:49.713200 3482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:49.713794 3482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:49.714610 3482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:49.714658 3482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:49.714710 3482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:49.714738 3482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:49.723367 3482 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:34101
I20260806 18:33:49.723416 3676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:34101 every 8 connection(s)
I20260806 18:33:49.724260 3482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:33:49.728993 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 3482
I20260806 18:33:49.729136 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:33:49.729403 7439 external_mini_cluster.cc:1428] Setting key ca9fc5b98e09a75f6df13c2b9563f805
Aug 06 18:33:49 dist-test-slave-5bn2 krb5kdc[2940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041229, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.195@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:33:49.737471 3062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:49.726091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:59517 (local address 127.7.67.254:36907)
0806 18:33:49.726245 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:49.726250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:49.727120 (+ 870us) server_negotiation.cc:407] Connection header received
0806 18:33:49.728503 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:49.728506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:49.728552 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:49.728623 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:49.729859 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.730516 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.731861 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.732073 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.732152 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:49.734500 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:49.734503 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:49.734505 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:49.734528 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:49.735820 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:49.736300 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:49.736302 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:49.736304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:49.736371 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:49.736622 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:49.736624 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:49.736625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:49.736778 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:49.736845 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:49.737258 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 18:33:49.737347 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:33:49.738277 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:49.726474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:36907 (local address 127.7.67.195:59517)
0806 18:33:49.726970 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:49.727015 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:49.728302 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:49.728775 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:49.728781 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:49.729170 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:49.729685 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:49.729701 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.730648 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.730652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:49.731748 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:49.731758 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.732229 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.732233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:49.732391 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:49.732905 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:33:49.732920 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:49.734348 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:49.735954 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:49.735959 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:49.735971 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:49.736222 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:49.736432 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:33:49.736434 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:33:49.736435 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:33:49.736559 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:33:49.736844 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:49.736849 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:49.737076 (+ 227us) client_negotiation.cc:769] Sending connection context
0806 18:33:49.737225 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 18:33:49.737473 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":117,"threads_started":1}
I20260806 18:33:49.739362 3677 heartbeater.cc:347] Connected to a master server at 127.7.67.254:36907
I20260806 18:33:49.739584 3677 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:49.740060 3677 heartbeater.cc:511] Master 127.7.67.254:36907 requested a full tablet report, sending...
I20260806 18:33:49.741031 3012 ts_manager.cc:194] Registered new tserver with Master: 4618d8673e43475fbbd726441c431e6b (127.7.67.195:34101)
I20260806 18:33:49.741571 3012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.195@KRBTEST.COM'} at 127.7.67.195:59517
I20260806 18:33:49.743319 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:33:49.752190 3062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:49.745459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47286 (local address 127.7.67.254:36907)
0806 18:33:49.745575 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:49.745578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:49.745792 (+ 214us) server_negotiation.cc:407] Connection header received
0806 18:33:49.745876 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:49.745878 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:49.745917 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:49.745986 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:49.746620 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.747178 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.747973 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.748181 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.748247 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:49.749423 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:49.749426 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:49.749428 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:49.749449 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:49.750729 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:49.751183 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:49.751186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:49.751187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:49.751261 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:49.751470 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:49.751472 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:49.751473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:49.751673 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:49.751770 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:49.751967 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 18:33:49.752067 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:33:49.754989 3012 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:33:49.757908 3012 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:33:49.771970 3703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:49.766839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57768 (local address 127.7.67.195:34101)
0806 18:33:49.767237 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:49.767249 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:49.767271 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:33:49.767348 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:49.767353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:49.767546 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:49.767717 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:49.768225 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.769016 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.770507 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.771256 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.771414 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:49.771580 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:49.771723 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 18:33:49.771817 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":139,"threads_started":1}
I20260806 18:33:49.771966 3699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:49.766124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:34101 (local address 127.0.0.1:57768)
0806 18:33:49.766731 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:49.766743 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:49.766829 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:49.767903 (+ 1074us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:49.767908 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:49.767942 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:49.768092 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:49.768100 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.769155 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.769158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:49.770365 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:49.770377 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.771422 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.771426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:49.771533 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:49.771573 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 18:33:49.771682 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 18:33:49.771810 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":5,"spinlock_wait_cycles":7936,"thread_start_us":101,"threads_started":1}
I20260806 18:33:49.772399 3702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:49.766743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:42703 (local address 127.0.0.1:52140)
0806 18:33:49.767295 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:49.767310 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:49.767395 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:49.768083 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:49.768085 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:49.768101 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:49.768170 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:49.768175 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.769576 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.769579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:49.770813 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:49.770824 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.771859 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.771863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:49.771959 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:49.771983 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:33:49.772168 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 18:33:49.772240 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":49,"thread_start_us":124,"threads_started":1}
I20260806 18:33:49.772485 3701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:49.766168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51652 (local address 127.7.67.193:40699)
0806 18:33:49.766555 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:49.766572 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:49.766602 (+ 30us) server_negotiation.cc:407] Connection header received
0806 18:33:49.766914 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:49.766920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:49.767556 (+ 636us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:49.767715 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:49.768239 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.769430 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.770890 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.771714 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.771823 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:49.771963 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:49.772237 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 18:33:49.772335 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":100,"threads_started":1}
I20260806 18:33:49.772401 3700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:49.766012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:40699 (local address 127.0.0.1:51652)
0806 18:33:49.766493 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:49.766542 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:49.766669 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:49.767904 (+ 1235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:49.767910 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:49.767946 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:49.768108 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:49.768116 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.769606 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.769609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:49.770751 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:49.770762 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.772021 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.772025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:49.772105 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:49.772127 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:33:49.772188 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 18:33:49.772239 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":92,"spinlock_wait_cycles":4736,"thread_start_us":90,"threads_started":1}
I20260806 18:33:49.773198 3704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:49.767152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52140 (local address 127.7.67.194:42703)
0806 18:33:49.767639 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:49.767650 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:49.767672 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:33:49.767741 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:49.767747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:49.767936 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:49.768088 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:49.768252 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.769431 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.770979 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:49.771722 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:49.772770 (+ 1048us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:49.772914 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:49.772967 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 18:33:49.773052 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":109,"threads_started":1}
I20260806 18:33:49.775568 3386 tablet_service.cc:1467] Processing CreateTablet for tablet 435c9a8d9c554666b46aae25abc3c6c5 (DEFAULT_TABLE table=test-table [id=5f49f4655d144408b6dccc8a7ad322ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:49.775540 3578 tablet_service.cc:1467] Processing CreateTablet for tablet 435c9a8d9c554666b46aae25abc3c6c5 (DEFAULT_TABLE table=test-table [id=5f49f4655d144408b6dccc8a7ad322ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:49.775540 3191 tablet_service.cc:1467] Processing CreateTablet for tablet 435c9a8d9c554666b46aae25abc3c6c5 (DEFAULT_TABLE table=test-table [id=5f49f4655d144408b6dccc8a7ad322ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:49.776536 3386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 435c9a8d9c554666b46aae25abc3c6c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:49.776582 3578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 435c9a8d9c554666b46aae25abc3c6c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:49.776582 3191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 435c9a8d9c554666b46aae25abc3c6c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:49.783193 3705 tablet_bootstrap.cc:492] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db: Bootstrap starting.
I20260806 18:33:49.784327 3706 tablet_bootstrap.cc:492] T 435c9a8d9c554666b46aae25abc3c6c5 P 724ac09266c54619b8d55a6815c27a8c: Bootstrap starting.
I20260806 18:33:49.784561 3707 tablet_bootstrap.cc:492] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b: Bootstrap starting.
I20260806 18:33:49.785339 3705 tablet_bootstrap.cc:654] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:49.786435 3705 log.cc:824] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:49.786514 3706 tablet_bootstrap.cc:654] T 435c9a8d9c554666b46aae25abc3c6c5 P 724ac09266c54619b8d55a6815c27a8c: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:49.786514 3707 tablet_bootstrap.cc:654] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:49.787834 3707 log.cc:824] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:49.787834 3706 log.cc:824] T 435c9a8d9c554666b46aae25abc3c6c5 P 724ac09266c54619b8d55a6815c27a8c: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:49.788714 3705 tablet_bootstrap.cc:492] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db: No bootstrap required, opened a new log
I20260806 18:33:49.788918 3705 ts_tablet_manager.cc:1397] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:33:49.789772 3707 tablet_bootstrap.cc:492] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b: No bootstrap required, opened a new log
I20260806 18:33:49.789768 3706 tablet_bootstrap.cc:492] T 435c9a8d9c554666b46aae25abc3c6c5 P 724ac09266c54619b8d55a6815c27a8c: No bootstrap required, opened a new log
I20260806 18:33:49.789953 3707 ts_tablet_manager.cc:1397] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:33:49.789953 3706 ts_tablet_manager.cc:1397] T 435c9a8d9c554666b46aae25abc3c6c5 P 724ac09266c54619b8d55a6815c27a8c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:33:49.791970 3705 raft_consensus.cc:348] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724ac09266c54619b8d55a6815c27a8c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 40699 } } peers { permanent_uuid: "4618d8673e43475fbbd726441c431e6b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 34101 } } peers { permanent_uuid: "0d6ff63ef5124852a3c51079547e64db" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42703 } }
I20260806 18:33:49.792225 3705 raft_consensus.cc:374] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:49.792291 3705 raft_consensus.cc:729] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d6ff63ef5124852a3c51079547e64db, State: Initialized, Role: FOLLOWER
I20260806 18:33:49.792732 3705 consensus_queue.cc:260] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724ac09266c54619b8d55a6815c27a8c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 40699 } } peers { permanent_uuid: "4618d8673e43475fbbd726441c431e6b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 34101 } } peers { permanent_uuid: "0d6ff63ef5124852a3c51079547e64db" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42703 } }
I20260806 18:33:49.793403 3476 heartbeater.cc:503] Master 127.7.67.254:36907 was elected leader, sending a full tablet report...
I20260806 18:33:49.793783 3705 ts_tablet_manager.cc:1428] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:33:49.793761 3706 raft_consensus.cc:348] T 435c9a8d9c554666b46aae25abc3c6c5 P 724ac09266c54619b8d55a6815c27a8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724ac09266c54619b8d55a6815c27a8c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 40699 } } peers { permanent_uuid: "4618d8673e43475fbbd726441c431e6b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 34101 } } peers { permanent_uuid: "0d6ff63ef5124852a3c51079547e64db" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42703 } }
I20260806 18:33:49.794013 3706 raft_consensus.cc:374] T 435c9a8d9c554666b46aae25abc3c6c5 P 724ac09266c54619b8d55a6815c27a8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:49.794071 3706 raft_consensus.cc:729] T 435c9a8d9c554666b46aae25abc3c6c5 P 724ac09266c54619b8d55a6815c27a8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 724ac09266c54619b8d55a6815c27a8c, State: Initialized, Role: FOLLOWER
I20260806 18:33:49.794494 3706 consensus_queue.cc:260] T 435c9a8d9c554666b46aae25abc3c6c5 P 724ac09266c54619b8d55a6815c27a8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724ac09266c54619b8d55a6815c27a8c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 40699 } } peers { permanent_uuid: "4618d8673e43475fbbd726441c431e6b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 34101 } } peers { permanent_uuid: "0d6ff63ef5124852a3c51079547e64db" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42703 } }
I20260806 18:33:49.795214 3283 heartbeater.cc:503] Master 127.7.67.254:36907 was elected leader, sending a full tablet report...
I20260806 18:33:49.795539 3706 ts_tablet_manager.cc:1428] T 435c9a8d9c554666b46aae25abc3c6c5 P 724ac09266c54619b8d55a6815c27a8c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:33:49.796097 3707 raft_consensus.cc:348] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724ac09266c54619b8d55a6815c27a8c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 40699 } } peers { permanent_uuid: "4618d8673e43475fbbd726441c431e6b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 34101 } } peers { permanent_uuid: "0d6ff63ef5124852a3c51079547e64db" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42703 } }
I20260806 18:33:49.796361 3707 raft_consensus.cc:374] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:49.796411 3707 raft_consensus.cc:729] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4618d8673e43475fbbd726441c431e6b, State: Initialized, Role: FOLLOWER
I20260806 18:33:49.796862 3707 consensus_queue.cc:260] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724ac09266c54619b8d55a6815c27a8c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 40699 } } peers { permanent_uuid: "4618d8673e43475fbbd726441c431e6b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 34101 } } peers { permanent_uuid: "0d6ff63ef5124852a3c51079547e64db" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42703 } }
I20260806 18:33:49.798241 3707 ts_tablet_manager.cc:1428] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 18:33:49.798554 3677 heartbeater.cc:503] Master 127.7.67.254:36907 was elected leader, sending a full tablet report...
W20260806 18:33:49.975294 3678 tablet.cc:2369] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:33:50.009850 3477 tablet.cc:2369] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:33:50.029678 3718 raft_consensus.cc:482] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:33:50.029930 3718 raft_consensus.cc:504] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724ac09266c54619b8d55a6815c27a8c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 40699 } } peers { permanent_uuid: "4618d8673e43475fbbd726441c431e6b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 34101 } } peers { permanent_uuid: "0d6ff63ef5124852a3c51079547e64db" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42703 } }
I20260806 18:33:50.031236 3718 leader_election.cc:288] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 724ac09266c54619b8d55a6815c27a8c (127.7.67.193:40699), 0d6ff63ef5124852a3c51079547e64db (127.7.67.194:42703)
I20260806 18:33:50.034811 3701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:50.031611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:60527 (local address 127.7.67.193:40699)
0806 18:33:50.031747 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:50.031752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:50.031768 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:33:50.031963 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:50.031967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:50.032022 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:50.032106 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:50.032374 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.032888 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.033906 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.034287 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.034405 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:50.034461 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:50.034623 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 18:33:50.034691 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:33:50.034811 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:50.031523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:40699 (local address 127.7.67.195:60527)
0806 18:33:50.031668 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:50.031685 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:50.031809 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:50.032129 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:50.032132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:50.032150 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:50.032261 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:50.032268 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.033000 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.033003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:50.033773 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:50.033781 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.034405 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.034409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:50.034515 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:50.034534 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:33:50.034598 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 18:33:50.034666 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 18:33:50.035290 3721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:50.031807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:42703 (local address 127.7.67.195:56267)
0806 18:33:50.032217 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:50.032228 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:50.032324 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:50.032544 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:50.032546 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:50.032576 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:50.032695 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:50.032700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.033407 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.033410 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:50.034217 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:50.034225 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.034871 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.034875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:50.034981 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:50.035006 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:33:50.035080 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:33:50.035143 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":11,"thread_start_us":138,"threads_started":1}
I20260806 18:33:50.035483 3704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:50.031966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:56267 (local address 127.7.67.194:42703)
0806 18:33:50.032103 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:50.032107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:50.032300 (+ 193us) server_negotiation.cc:407] Connection header received
0806 18:33:50.032412 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:50.032415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:50.032459 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:50.032518 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:50.032792 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.033298 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.034336 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.034721 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.034827 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:50.034903 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:50.035296 (+ 393us) server_negotiation.cc:299] Negotiation successful
0806 18:33:50.035378 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:33:50.035576 3221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "435c9a8d9c554666b46aae25abc3c6c5" candidate_uuid: "4618d8673e43475fbbd726441c431e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "724ac09266c54619b8d55a6815c27a8c" is_pre_election: true
I20260806 18:33:50.035925 3221 raft_consensus.cc:2454] T 435c9a8d9c554666b46aae25abc3c6c5 P 724ac09266c54619b8d55a6815c27a8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4618d8673e43475fbbd726441c431e6b in term 0.
I20260806 18:33:50.036144 3415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "435c9a8d9c554666b46aae25abc3c6c5" candidate_uuid: "4618d8673e43475fbbd726441c431e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d6ff63ef5124852a3c51079547e64db" is_pre_election: true
I20260806 18:33:50.036420 3415 raft_consensus.cc:2454] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4618d8673e43475fbbd726441c431e6b in term 0.
I20260806 18:33:50.036420 3509 leader_election.cc:302] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4618d8673e43475fbbd726441c431e6b, 724ac09266c54619b8d55a6815c27a8c; no voters:
I20260806 18:33:50.036743 3718 raft_consensus.cc:2781] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:33:50.036851 3718 raft_consensus.cc:482] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:33:50.036888 3718 raft_consensus.cc:3037] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:50.037930 3718 raft_consensus.cc:504] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724ac09266c54619b8d55a6815c27a8c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 40699 } } peers { permanent_uuid: "4618d8673e43475fbbd726441c431e6b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 34101 } } peers { permanent_uuid: "0d6ff63ef5124852a3c51079547e64db" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42703 } }
I20260806 18:33:50.038251 3718 leader_election.cc:288] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [CANDIDATE]: Term 1 election: Requested vote from peers 724ac09266c54619b8d55a6815c27a8c (127.7.67.193:40699), 0d6ff63ef5124852a3c51079547e64db (127.7.67.194:42703)
I20260806 18:33:50.038640 3415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "435c9a8d9c554666b46aae25abc3c6c5" candidate_uuid: "4618d8673e43475fbbd726441c431e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d6ff63ef5124852a3c51079547e64db"
I20260806 18:33:50.038621 3221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "435c9a8d9c554666b46aae25abc3c6c5" candidate_uuid: "4618d8673e43475fbbd726441c431e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "724ac09266c54619b8d55a6815c27a8c"
I20260806 18:33:50.038756 3415 raft_consensus.cc:3037] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:50.038756 3221 raft_consensus.cc:3037] T 435c9a8d9c554666b46aae25abc3c6c5 P 724ac09266c54619b8d55a6815c27a8c [term 0 FOLLOWER]: Advancing to term 1
W20260806 18:33:50.038981 3284 tablet.cc:2369] T 435c9a8d9c554666b46aae25abc3c6c5 P 724ac09266c54619b8d55a6815c27a8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:33:50.040081 3415 raft_consensus.cc:2454] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4618d8673e43475fbbd726441c431e6b in term 1.
I20260806 18:33:50.040086 3221 raft_consensus.cc:2454] T 435c9a8d9c554666b46aae25abc3c6c5 P 724ac09266c54619b8d55a6815c27a8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4618d8673e43475fbbd726441c431e6b in term 1.
I20260806 18:33:50.040380 3509 leader_election.cc:302] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d6ff63ef5124852a3c51079547e64db, 4618d8673e43475fbbd726441c431e6b; no voters:
I20260806 18:33:50.040534 3718 raft_consensus.cc:2781] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:33:50.040758 3718 raft_consensus.cc:686] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [term 1 LEADER]: Becoming Leader. State: Replica: 4618d8673e43475fbbd726441c431e6b, State: Running, Role: LEADER
I20260806 18:33:50.041097 3718 consensus_queue.cc:237] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724ac09266c54619b8d55a6815c27a8c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 40699 } } peers { permanent_uuid: "4618d8673e43475fbbd726441c431e6b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 34101 } } peers { permanent_uuid: "0d6ff63ef5124852a3c51079547e64db" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42703 } }
I20260806 18:33:50.044376 3012 catalog_manager.cc:5495] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4618d8673e43475fbbd726441c431e6b (127.7.67.195). New cstate: current_term: 1 leader_uuid: "4618d8673e43475fbbd726441c431e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724ac09266c54619b8d55a6815c27a8c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 40699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4618d8673e43475fbbd726441c431e6b" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 34101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d6ff63ef5124852a3c51079547e64db" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42703 } health_report { overall_health: UNKNOWN } } }
I20260806 18:33:50.065431 3703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:50.061689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57772 (local address 127.7.67.195:34101)
0806 18:33:50.061870 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:50.061877 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:50.061896 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:33:50.061988 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:50.061990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:50.062050 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:50.062153 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:33:50.062435 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.062996 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.063922 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.064139 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.064239 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:50.064562 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:33:50.065003 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:33:50.065125 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:50.065201 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 18:33:50.065269 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 18:33:50.071801 3415 raft_consensus.cc:1261] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db [term 1 FOLLOWER]: Refusing update from remote peer 4618d8673e43475fbbd726441c431e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:33:50.071801 3221 raft_consensus.cc:1261] T 435c9a8d9c554666b46aae25abc3c6c5 P 724ac09266c54619b8d55a6815c27a8c [term 1 FOLLOWER]: Refusing update from remote peer 4618d8673e43475fbbd726441c431e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:33:50.072572 3718 consensus_queue.cc:1035] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "724ac09266c54619b8d55a6815c27a8c" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 40699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:33:50.072820 3722 consensus_queue.cc:1035] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d6ff63ef5124852a3c51079547e64db" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 42703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:33:50.081390 3731 mvcc.cc:205] Tried to move back new op lower bound from 7315624878365450240 to 7315624878250713088. Current Snapshot: MvccSnapshot[applied={T|T < 7315624878365450240}]
I20260806 18:33:50.089193 3012 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:33:50.089367 3012 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:33:50.091851 3012 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c299b9456784748859a2087bb76b429: Sending DeleteTablet for 3 replicas of tablet 435c9a8d9c554666b46aae25abc3c6c5
I20260806 18:33:50.092692 3578 tablet_service.cc:1514] Processing DeleteTablet for tablet 435c9a8d9c554666b46aae25abc3c6c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:50 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:57768
I20260806 18:33:50.092716 3191 tablet_service.cc:1514] Processing DeleteTablet for tablet 435c9a8d9c554666b46aae25abc3c6c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:50 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:51652
I20260806 18:33:50.093007 3386 tablet_service.cc:1514] Processing DeleteTablet for tablet 435c9a8d9c554666b46aae25abc3c6c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:50 UTC) from {username='kudu', principal='kudu/127.7.67.254@KRBTEST.COM'} at 127.0.0.1:52140
I20260806 18:33:50.093254 3743 tablet_replica.cc:330] T 435c9a8d9c554666b46aae25abc3c6c5 P 724ac09266c54619b8d55a6815c27a8c: stopping tablet replica
I20260806 18:33:50.093576 3743 raft_consensus.cc:2229] T 435c9a8d9c554666b46aae25abc3c6c5 P 724ac09266c54619b8d55a6815c27a8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:33:50.093724 3742 tablet_replica.cc:330] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b: stopping tablet replica
I20260806 18:33:50.093905 3743 raft_consensus.cc:2258] T 435c9a8d9c554666b46aae25abc3c6c5 P 724ac09266c54619b8d55a6815c27a8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:33:50.094010 3742 raft_consensus.cc:2229] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:33:50.094071 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 3095
I20260806 18:33:50.099773 3742 raft_consensus.cc:2258] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:33:50.102913 3742 ts_tablet_manager.cc:1905] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:33:50.106017 3742 ts_tablet_manager.cc:1918] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:33:50.106145 3742 log.cc:1196] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/435c9a8d9c554666b46aae25abc3c6c5
I20260806 18:33:50.106523 3742 ts_tablet_manager.cc:1939] T 435c9a8d9c554666b46aae25abc3c6c5 P 4618d8673e43475fbbd726441c431e6b: Deleting consensus metadata
I20260806 18:33:50.107123 3744 tablet_replica.cc:330] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db: stopping tablet replica
I20260806 18:33:50.107568 3744 raft_consensus.cc:2229] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:33:50.107936 2994 catalog_manager.cc:4841] TS 4618d8673e43475fbbd726441c431e6b (127.7.67.195:34101): tablet 435c9a8d9c554666b46aae25abc3c6c5 (table test-table [id=5f49f4655d144408b6dccc8a7ad322ca]) successfully deleted
I20260806 18:33:50.107998 3744 raft_consensus.cc:2258] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:33:50.109844 3744 ts_tablet_manager.cc:1905] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 18:33:50.112237 2990 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.193:40699 (error 108)
I20260806 18:33:50.112684 3744 ts_tablet_manager.cc:1918] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:33:50.112794 3744 log.cc:1196] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786040973123132-7439-0/minicluster-data/ts-1/wal/wals/435c9a8d9c554666b46aae25abc3c6c5
I20260806 18:33:50.113108 3744 ts_tablet_manager.cc:1939] T 435c9a8d9c554666b46aae25abc3c6c5 P 0d6ff63ef5124852a3c51079547e64db: Deleting consensus metadata
I20260806 18:33:50.113183 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 3289
I20260806 18:33:50.113626 3699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:50.113220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:40699 (local address 127.0.0.1:51656)
0806 18:33:50.113461 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:50.113532 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
W20260806 18:33:50.124284 2990 catalog_manager.cc:4584] TS 724ac09266c54619b8d55a6815c27a8c (127.7.67.193:40699): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 435c9a8d9c554666b46aae25abc3c6c5: Network error: Client connection negotiation failed: client connection to 127.7.67.193:40699: connect: Connection refused (error 111)
I20260806 18:33:50.130398 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 3482
I20260806 18:33:50.130672 3699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:50.130352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:42703 (local address 127.0.0.1:52156)
0806 18:33:50.130509 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:50.130573 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.194:42703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 18:33:50.140776 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 2961
2026-08-06T18:33:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2595 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T18:33:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:33:50Z Disabled control of system clock
I20260806 18:33:50.176185 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:45415
--webserver_interface=127.7.67.254
--webserver_port=0
--builtin_ntp_servers=127.7.67.212:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:45415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:33:50.289304 3750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:50.289624 3750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:50.289661 3750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:50.293802 3750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:33:50.293879 3750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:50.293893 3750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:50.293912 3750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:33:50.293932 3750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:33:50.298946 3750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:45415
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:45415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:50.300097 3750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:50.301580 3750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:50.310664 3759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:50.310672 3758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:50.310672 3761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:50.311087 3750 server_base.cc:1034] running on GCE node
I20260806 18:33:50.311717 3750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:50.312870 3750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:50.314029 3750 hybrid_clock.cc:648] HybridClock initialized: now 1786041230314005 us; error 36 us; skew 500 ppm
I20260806 18:33:50.315972 3750 webserver.cc:460] Webserver started at http://127.7.67.254:46229/ using document root <none> and password file <none>
I20260806 18:33:50.316582 3750 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:50.316650 3750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:50.316856 3750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:50.319090 3750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "19c38a89ce3e4b83a5e5ac585ca2a7dd"
format_stamp: "Formatted at 2026-08-06 18:33:50 on dist-test-slave-5bn2"
server_key: "012b00d228efb489e46bc695e655805f"
server_key_iv: "57387e9b156a383e62678ffd82982f2c"
server_key_version: "encryptionkey@0"
I20260806 18:33:50.319679 3750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "19c38a89ce3e4b83a5e5ac585ca2a7dd"
format_stamp: "Formatted at 2026-08-06 18:33:50 on dist-test-slave-5bn2"
server_key: "012b00d228efb489e46bc695e655805f"
server_key_iv: "57387e9b156a383e62678ffd82982f2c"
server_key_version: "encryptionkey@0"
I20260806 18:33:50.323645 3750 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:33:50.326179 3768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:50.327497 3750 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:33:50.327636 3750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "19c38a89ce3e4b83a5e5ac585ca2a7dd"
format_stamp: "Formatted at 2026-08-06 18:33:50 on dist-test-slave-5bn2"
server_key: "012b00d228efb489e46bc695e655805f"
server_key_iv: "57387e9b156a383e62678ffd82982f2c"
server_key_version: "encryptionkey@0"
I20260806 18:33:50.327744 3750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:50.343688 3750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:50.344403 3750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:50.344596 3750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:50.351670 3750 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:45415
I20260806 18:33:50.351692 3846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:45415 every 8 connection(s)
I20260806 18:33:50.352680 3750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:33:50.355902 3847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:50.361636 3847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd: Bootstrap starting.
I20260806 18:33:50.362234 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 3750
I20260806 18:33:50.362358 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:33:50.362634 7439 external_mini_cluster.cc:1428] Setting key 2b012af802c59ea3ce41ecbfcc7faa75
I20260806 18:33:50.364602 3847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:50.365362 3847 log.cc:824] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:50.367538 3847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd: No bootstrap required, opened a new log
I20260806 18:33:50.371187 3847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c38a89ce3e4b83a5e5ac585ca2a7dd" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45415 } }
I20260806 18:33:50.371479 3847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:50.371546 3847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19c38a89ce3e4b83a5e5ac585ca2a7dd, State: Initialized, Role: FOLLOWER
I20260806 18:33:50.371997 3847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c38a89ce3e4b83a5e5ac585ca2a7dd" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45415 } }
I20260806 18:33:50.372133 3847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:33:50.372138 3852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:50.364056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41218 (local address 127.7.67.254:45415)
0806 18:33:50.364760 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:50.364776 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:50.364814 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:33:50.365454 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:50.365471 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:50.366014 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:50.366382 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:50.366831 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.367905 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.368826 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.369303 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.369452 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:50.370287 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:50.370292 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:50.370305 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:50.370708 (+ 403us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:50.370766 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:50.370786 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:50.370899 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:50.371043 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 18:33:50.371313 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":138,"threads_started":1}
I20260806 18:33:50.372210 3847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:33:50.372308 3847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:50.373438 3847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c38a89ce3e4b83a5e5ac585ca2a7dd" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45415 } }
I20260806 18:33:50.373764 3847 leader_election.cc:302] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19c38a89ce3e4b83a5e5ac585ca2a7dd; no voters:
I20260806 18:33:50.374060 3847 leader_election.cc:288] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:33:50.374143 3856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:33:50.374322 3856 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd [term 1 LEADER]: Becoming Leader. State: Replica: 19c38a89ce3e4b83a5e5ac585ca2a7dd, State: Running, Role: LEADER
I20260806 18:33:50.374629 3856 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c38a89ce3e4b83a5e5ac585ca2a7dd" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45415 } }
I20260806 18:33:50.375391 3847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:33:50.376380 3858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19c38a89ce3e4b83a5e5ac585ca2a7dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c38a89ce3e4b83a5e5ac585ca2a7dd" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45415 } } }
I20260806 18:33:50.376515 3858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:50.376822 3859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19c38a89ce3e4b83a5e5ac585ca2a7dd. Latest consensus state: current_term: 1 leader_uuid: "19c38a89ce3e4b83a5e5ac585ca2a7dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c38a89ce3e4b83a5e5ac585ca2a7dd" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45415 } } }
I20260806 18:33:50.376926 3859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:50.376986 3870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:33:50.380240 3870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:33:50.385995 3870 catalog_manager.cc:1347] Generated new cluster ID: 1c82a50d595b45a48e3bc6250aa74a6e
I20260806 18:33:50.386091 3870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:33:50.398459 3870 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:33:50.399822 3870 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:33:50.417563 3870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd: Generated new TSK 0
I20260806 18:33:50.418553 3870 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:33:50.426083 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.193:0
--local_ip_for_outbound_sockets=127.7.67.193
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:45415
--builtin_ntp_servers=127.7.67.212:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:33:50.544809 3881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:50.545105 3881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:50.545146 3881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:50.549288 3881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:50.549362 3881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:50.549431 3881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.193
I20260806 18:33:50.554335 3881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.193
--webserver_port=0
--tserver_master_addrs=127.7.67.254:45415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.193
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:50.555560 3881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:50.557005 3881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:50.566658 3887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:50.566650 3891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:50.566650 3886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:50.567083 3881 server_base.cc:1034] running on GCE node
I20260806 18:33:50.567484 3881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:50.568116 3881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:50.569278 3881 hybrid_clock.cc:648] HybridClock initialized: now 1786041230569252 us; error 35 us; skew 500 ppm
I20260806 18:33:50.571239 3881 webserver.cc:460] Webserver started at http://127.7.67.193:42931/ using document root <none> and password file <none>
I20260806 18:33:50.571825 3881 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:50.571893 3881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:50.572119 3881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:50.574250 3881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-0/data/instance:
uuid: "2fd60413838a4b549e10db26757efa7b"
format_stamp: "Formatted at 2026-08-06 18:33:50 on dist-test-slave-5bn2"
server_key: "3d2406d9a24d5da1c84e23bcd314975a"
server_key_iv: "5259cfdbf3000829520f61cdae408c74"
server_key_version: "encryptionkey@0"
I20260806 18:33:50.574805 3881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance:
uuid: "2fd60413838a4b549e10db26757efa7b"
format_stamp: "Formatted at 2026-08-06 18:33:50 on dist-test-slave-5bn2"
server_key: "3d2406d9a24d5da1c84e23bcd314975a"
server_key_iv: "5259cfdbf3000829520f61cdae408c74"
server_key_version: "encryptionkey@0"
I20260806 18:33:50.579499 3881 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:33:50.581975 3900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:50.583262 3881 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:33:50.583379 3881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-0/wal
uuid: "2fd60413838a4b549e10db26757efa7b"
format_stamp: "Formatted at 2026-08-06 18:33:50 on dist-test-slave-5bn2"
server_key: "3d2406d9a24d5da1c84e23bcd314975a"
server_key_iv: "5259cfdbf3000829520f61cdae408c74"
server_key_version: "encryptionkey@0"
I20260806 18:33:50.583482 3881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:50.601097 3881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:50.601850 3881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:50.602032 3881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:50.602655 3881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:50.603701 3881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:50.603752 3881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:50.603809 3881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:50.603840 3881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:50.612583 3881 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.193:44361
I20260806 18:33:50.612649 4055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.193:44361 every 8 connection(s)
I20260806 18:33:50.613503 3881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-0/data/info.pb
I20260806 18:33:50.622676 3852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:50.615389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.193:52325 (local address 127.7.67.254:45415)
0806 18:33:50.615532 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:50.615536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:50.616618 (+ 1082us) server_negotiation.cc:407] Connection header received
0806 18:33:50.618019 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:50.618021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:50.618072 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:50.618136 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:50.619160 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.619847 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.620976 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.621204 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.621284 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:50.622268 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:50.622271 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:50.622274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:50.622287 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:50.622306 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:50.622314 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:50.622370 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:50.622490 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 18:33:50.622544 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:33:50.623545 4058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:50.615765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:45415 (local address 127.7.67.193:52325)
0806 18:33:50.616462 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:50.616504 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:50.617812 (+ 1308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:50.618292 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:50.618299 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:50.618383 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:50.618996 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:50.619017 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.619970 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.619972 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:50.620861 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:50.620871 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.621328 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.621330 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:50.621482 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:50.622022 (+ 540us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:33:50.622024 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:50.622064 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:33:50.622129 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:50.622375 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:50.622380 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:50.622384 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:33:50.622483 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 18:33:50.622708 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":130,"threads_started":1}
I20260806 18:33:50.623853 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 3881
I20260806 18:33:50.623998 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-0/wal/instance
I20260806 18:33:50.624295 7439 external_mini_cluster.cc:1428] Setting key 170e2cf38867778be2640996f93ebd70
I20260806 18:33:50.624846 4056 heartbeater.cc:347] Connected to a master server at 127.7.67.254:45415
I20260806 18:33:50.625089 4056 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:50.625622 4056 heartbeater.cc:511] Master 127.7.67.254:45415 requested a full tablet report, sending...
I20260806 18:33:50.627127 3788 ts_manager.cc:194] Registered new tserver with Master: 2fd60413838a4b549e10db26757efa7b (127.7.67.193:44361)
I20260806 18:33:50.627492 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.194:0
--local_ip_for_outbound_sockets=127.7.67.194
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:45415
--builtin_ntp_servers=127.7.67.212:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:33:50.628379 3788 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.193:52325
W20260806 18:33:50.743001 4060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:50.743288 4060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:50.743330 4060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:50.747370 4060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:50.747433 4060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:50.747488 4060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.194
I20260806 18:33:50.752290 4060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.194
--webserver_port=0
--tserver_master_addrs=127.7.67.254:45415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.194
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:50.753413 4060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:50.754882 4060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:33:50.764730 4060 server_base.cc:1034] running on GCE node
W20260806 18:33:50.764714 4071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:50.764688 4073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:50.764739 4069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:50.765435 4060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:50.766034 4060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:50.767196 4060 hybrid_clock.cc:648] HybridClock initialized: now 1786041230767178 us; error 30 us; skew 500 ppm
I20260806 18:33:50.769186 4060 webserver.cc:460] Webserver started at http://127.7.67.194:44907/ using document root <none> and password file <none>
I20260806 18:33:50.769876 4060 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:50.769946 4060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:50.770134 4060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:50.772264 4060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-1/data/instance:
uuid: "6ccc954b995c49d893b9a21ea499404a"
format_stamp: "Formatted at 2026-08-06 18:33:50 on dist-test-slave-5bn2"
server_key: "62dcd3759f915ee06a2ba0663efb4c80"
server_key_iv: "4d072292cf0dc84140a5c8c421edc9f9"
server_key_version: "encryptionkey@0"
I20260806 18:33:50.772809 4060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance:
uuid: "6ccc954b995c49d893b9a21ea499404a"
format_stamp: "Formatted at 2026-08-06 18:33:50 on dist-test-slave-5bn2"
server_key: "62dcd3759f915ee06a2ba0663efb4c80"
server_key_iv: "4d072292cf0dc84140a5c8c421edc9f9"
server_key_version: "encryptionkey@0"
I20260806 18:33:50.776763 4060 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:33:50.779328 4081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:50.780568 4060 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:33:50.780690 4060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-1/wal
uuid: "6ccc954b995c49d893b9a21ea499404a"
format_stamp: "Formatted at 2026-08-06 18:33:50 on dist-test-slave-5bn2"
server_key: "62dcd3759f915ee06a2ba0663efb4c80"
server_key_iv: "4d072292cf0dc84140a5c8c421edc9f9"
server_key_version: "encryptionkey@0"
I20260806 18:33:50.780799 4060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:50.799343 4060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:50.800323 4060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:50.800563 4060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:50.801203 4060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:50.802286 4060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:50.802357 4060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:50.802416 4060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:50.802449 4060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:50.811090 4060 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.194:45601
I20260806 18:33:50.811142 4241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.194:45601 every 8 connection(s)
I20260806 18:33:50.812036 4060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-1/data/info.pb
I20260806 18:33:50.815263 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 4060
I20260806 18:33:50.815394 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-1/wal/instance
I20260806 18:33:50.815673 7439 external_mini_cluster.cc:1428] Setting key 48f6f95fb5bb74ca40018a4c14d166aa
I20260806 18:33:50.817657 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.195:0
--local_ip_for_outbound_sockets=127.7.67.195
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:45415
--builtin_ntp_servers=127.7.67.212:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:33:50.821585 3852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:50.813891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.194:41809 (local address 127.7.67.254:45415)
0806 18:33:50.814046 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:50.814050 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:50.815148 (+ 1098us) server_negotiation.cc:407] Connection header received
0806 18:33:50.816644 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:50.816648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:50.816704 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:50.816777 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:50.817714 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.818268 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.819601 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.819843 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.819941 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:50.821113 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:50.821117 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:50.821120 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:50.821133 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:50.821152 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:50.821160 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:50.821221 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:50.821364 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 18:33:50.821431 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 18:33:50.822499 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:50.814258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:45415 (local address 127.7.67.194:41809)
0806 18:33:50.814979 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:50.815039 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:50.816429 (+ 1390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:50.816969 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:50.816976 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:50.817071 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:50.817573 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:50.817587 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.818401 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.818404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:50.819423 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:50.819436 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:50.820005 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:50.820008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:50.820216 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:50.820844 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:33:50.820846 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:50.820895 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:33:50.820967 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:50.821230 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:50.821239 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:50.821245 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:33:50.821357 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 18:33:50.821648 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":133,"threads_started":1}
I20260806 18:33:50.823640 4243 heartbeater.cc:347] Connected to a master server at 127.7.67.254:45415
I20260806 18:33:50.823875 4243 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:50.824395 4243 heartbeater.cc:511] Master 127.7.67.254:45415 requested a full tablet report, sending...
I20260806 18:33:50.825456 3788 ts_manager.cc:194] Registered new tserver with Master: 6ccc954b995c49d893b9a21ea499404a (127.7.67.194:45601)
I20260806 18:33:50.826205 3788 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.194:41809
W20260806 18:33:50.931774 4248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:50.932054 4248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:50.932092 4248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:50.936028 4248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:50.936080 4248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:50.936134 4248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.195
I20260806 18:33:50.940804 4248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.195
--webserver_port=0
--tserver_master_addrs=127.7.67.254:45415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.195
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:50.941848 4248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:50.943243 4248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:50.952847 4256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:50.952852 4254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:50.952841 4258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:50.953068 4248 server_base.cc:1034] running on GCE node
I20260806 18:33:50.953596 4248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:50.954214 4248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:50.955371 4248 hybrid_clock.cc:648] HybridClock initialized: now 1786041230955343 us; error 40 us; skew 500 ppm
I20260806 18:33:50.957149 4248 webserver.cc:460] Webserver started at http://127.7.67.195:45625/ using document root <none> and password file <none>
I20260806 18:33:50.957719 4248 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:50.957787 4248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:50.957993 4248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:50.960105 4248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-2/data/instance:
uuid: "328962dc02db47a38c18ae7976104fb2"
format_stamp: "Formatted at 2026-08-06 18:33:50 on dist-test-slave-5bn2"
server_key: "7698cfb908e856ae3290f4092953ddb5"
server_key_iv: "a2dafaf34da0d9ef9bfc5fd5f04eb128"
server_key_version: "encryptionkey@0"
I20260806 18:33:50.960700 4248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance:
uuid: "328962dc02db47a38c18ae7976104fb2"
format_stamp: "Formatted at 2026-08-06 18:33:50 on dist-test-slave-5bn2"
server_key: "7698cfb908e856ae3290f4092953ddb5"
server_key_iv: "a2dafaf34da0d9ef9bfc5fd5f04eb128"
server_key_version: "encryptionkey@0"
I20260806 18:33:50.964542 4248 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:33:50.967059 4265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:50.968412 4248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:33:50.968528 4248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-2/wal
uuid: "328962dc02db47a38c18ae7976104fb2"
format_stamp: "Formatted at 2026-08-06 18:33:50 on dist-test-slave-5bn2"
server_key: "7698cfb908e856ae3290f4092953ddb5"
server_key_iv: "a2dafaf34da0d9ef9bfc5fd5f04eb128"
server_key_version: "encryptionkey@0"
I20260806 18:33:50.968648 4248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:50.989292 4248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:50.989990 4248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:50.990180 4248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:50.990778 4248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:33:50.991806 4248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:33:50.991854 4248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:50.991899 4248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:33:50.991927 4248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:51.000608 4248 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.195:37881
I20260806 18:33:51.000648 4423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.195:37881 every 8 connection(s)
I20260806 18:33:51.001530 4248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-2/data/info.pb
I20260806 18:33:51.004912 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 4248
I20260806 18:33:51.005039 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-2/wal/instance
I20260806 18:33:51.005369 7439 external_mini_cluster.cc:1428] Setting key 5cb2e59322c27c8418bade230379f79f
I20260806 18:33:51.011029 3852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:51.003433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:52449 (local address 127.7.67.254:45415)
0806 18:33:51.003582 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:51.003586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:51.004686 (+ 1100us) server_negotiation.cc:407] Connection header received
0806 18:33:51.006391 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:51.006393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:51.006442 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:51.006506 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:51.007673 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.008199 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.009319 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.009526 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.009603 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:51.010622 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:51.010626 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:51.010628 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:51.010640 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:51.010660 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:51.010668 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:51.010722 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:51.010831 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 18:33:51.010903 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:33:51.011873 4428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:51.003831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:45415 (local address 127.7.67.195:52449)
0806 18:33:51.004524 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:51.004569 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:51.006190 (+ 1621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:51.006702 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:51.006709 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:51.006797 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:33:51.007510 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:51.007552 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.008313 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.008316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:51.009225 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:51.009234 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.009636 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.009639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:51.009781 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:51.010361 (+ 580us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:33:51.010363 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:33:51.010404 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:33:51.010469 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:33:51.010728 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:33:51.010733 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:33:51.010736 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:33:51.010822 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 18:33:51.011088 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":124,"threads_started":1}
I20260806 18:33:51.012918 4424 heartbeater.cc:347] Connected to a master server at 127.7.67.254:45415
I20260806 18:33:51.013162 4424 heartbeater.cc:464] Registering TS with master...
I20260806 18:33:51.013695 4424 heartbeater.cc:511] Master 127.7.67.254:45415 requested a full tablet report, sending...
I20260806 18:33:51.014761 3788 ts_manager.cc:194] Registered new tserver with Master: 328962dc02db47a38c18ae7976104fb2 (127.7.67.195:37881)
I20260806 18:33:51.015403 3788 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.195:52449
I20260806 18:33:51.019430 7439 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:33:51.024960 3852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:51.021325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41226 (local address 127.7.67.254:45415)
0806 18:33:51.021461 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:51.021466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:51.021621 (+ 155us) server_negotiation.cc:407] Connection header received
0806 18:33:51.021734 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:51.021737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:51.021779 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:51.021843 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:51.022115 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.022615 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.023533 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.023745 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.023832 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:51.024570 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:51.024573 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:51.024575 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:51.024586 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:51.024603 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:51.024611 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:51.024670 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:51.024756 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 18:33:51.024811 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:33:51.027697 3788 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:33:51.030627 3788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:33:51.045230 4443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:51.039304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:45601 (local address 127.0.0.1:60388)
0806 18:33:51.040319 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:51.040373 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:51.040524 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:51.041220 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:51.041223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:51.041240 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:51.041334 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:51.041341 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.042363 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.042366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:51.043711 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:51.043723 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.044774 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.044778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:51.044953 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:51.044974 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:33:51.045027 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 18:33:51.045078 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"mutex_wait_us":73,"spinlock_wait_cycles":2048,"thread_start_us":111,"threads_started":1}
I20260806 18:33:51.045338 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:51.039789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60388 (local address 127.7.67.194:45601)
0806 18:33:51.040140 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:51.040152 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:51.040501 (+ 349us) server_negotiation.cc:407] Connection header received
0806 18:33:51.040679 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:51.040685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:51.040895 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:51.041059 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:51.041480 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.042245 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.043846 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.044629 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.044830 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:51.045010 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:51.045065 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 18:33:51.045166 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":140,"threads_started":1}
I20260806 18:33:51.045552 4445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:51.039793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40600 (local address 127.7.67.195:37881)
0806 18:33:51.040262 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:51.040274 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:51.040460 (+ 186us) server_negotiation.cc:407] Connection header received
0806 18:33:51.040681 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:51.040686 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:51.040904 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:51.041059 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:51.041338 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.042525 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.043981 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.044504 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.044615 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:51.044823 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:51.045299 (+ 476us) server_negotiation.cc:299] Negotiation successful
0806 18:33:51.045399 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":177,"threads_started":1}
I20260806 18:33:51.045230 4440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:51.039060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.195:37881 (local address 127.0.0.1:40600)
0806 18:33:51.040319 (+ 1259us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:51.040373 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:51.040524 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:51.041056 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:51.041062 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:51.041084 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:51.041227 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:51.041235 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.042641 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.042645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:51.043870 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:51.043879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.044629 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.044633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:51.044754 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:51.044813 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 18:33:51.044918 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 18:33:51.045039 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1084,"mutex_wait_us":32,"spinlock_wait_cycles":6528,"thread_start_us":117,"threads_started":1}
I20260806 18:33:51.046655 4442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:51.039109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38454 (local address 127.7.67.193:44361)
0806 18:33:51.039586 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:51.039598 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:51.040812 (+ 1214us) server_negotiation.cc:407] Connection header received
0806 18:33:51.040901 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:51.040904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:51.041083 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:51.041228 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:51.041586 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.042674 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.043995 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.044756 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.044839 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:51.045011 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:51.046389 (+ 1378us) server_negotiation.cc:299] Negotiation successful
0806 18:33:51.046492 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":162,"threads_started":1}
I20260806 18:33:51.046993 4441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:51.038969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:44361 (local address 127.0.0.1:38454)
0806 18:33:51.040706 (+ 1737us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:51.040718 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:51.040806 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:51.041355 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:51.041357 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:51.041375 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:51.041463 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:51.041469 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.042804 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.042808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:51.043861 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:51.043872 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.045075 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.045078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:51.045179 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:51.045202 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:33:51.045271 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 18:33:51.046842 (+ 1571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1151,"mutex_wait_us":25,"thread_start_us":79,"threads_started":1}
I20260806 18:33:51.048619 4148 tablet_service.cc:1467] Processing CreateTablet for tablet baa7de5581644fa4a8f9fa2a721562eb (DEFAULT_TABLE table=test-table [id=040010d3b2c24dd5aa50bc1edb89bce4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:51.048619 4330 tablet_service.cc:1467] Processing CreateTablet for tablet baa7de5581644fa4a8f9fa2a721562eb (DEFAULT_TABLE table=test-table [id=040010d3b2c24dd5aa50bc1edb89bce4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:51.049482 3969 tablet_service.cc:1467] Processing CreateTablet for tablet baa7de5581644fa4a8f9fa2a721562eb (DEFAULT_TABLE table=test-table [id=040010d3b2c24dd5aa50bc1edb89bce4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:33:51.049800 4148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baa7de5581644fa4a8f9fa2a721562eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:51.049800 4330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baa7de5581644fa4a8f9fa2a721562eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:51.050474 3969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baa7de5581644fa4a8f9fa2a721562eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:51.056627 4446 tablet_bootstrap.cc:492] T baa7de5581644fa4a8f9fa2a721562eb P 2fd60413838a4b549e10db26757efa7b: Bootstrap starting.
I20260806 18:33:51.057724 4448 tablet_bootstrap.cc:492] T baa7de5581644fa4a8f9fa2a721562eb P 6ccc954b995c49d893b9a21ea499404a: Bootstrap starting.
I20260806 18:33:51.058089 4449 tablet_bootstrap.cc:492] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2: Bootstrap starting.
I20260806 18:33:51.059072 4446 tablet_bootstrap.cc:654] T baa7de5581644fa4a8f9fa2a721562eb P 2fd60413838a4b549e10db26757efa7b: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:51.059803 4446 log.cc:824] T baa7de5581644fa4a8f9fa2a721562eb P 2fd60413838a4b549e10db26757efa7b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:51.060015 4449 tablet_bootstrap.cc:654] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:51.060015 4448 tablet_bootstrap.cc:654] T baa7de5581644fa4a8f9fa2a721562eb P 6ccc954b995c49d893b9a21ea499404a: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:51.060875 4448 log.cc:824] T baa7de5581644fa4a8f9fa2a721562eb P 6ccc954b995c49d893b9a21ea499404a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:51.060875 4449 log.cc:824] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:51.062469 4446 tablet_bootstrap.cc:492] T baa7de5581644fa4a8f9fa2a721562eb P 2fd60413838a4b549e10db26757efa7b: No bootstrap required, opened a new log
I20260806 18:33:51.062673 4446 ts_tablet_manager.cc:1397] T baa7de5581644fa4a8f9fa2a721562eb P 2fd60413838a4b549e10db26757efa7b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:33:51.062927 4448 tablet_bootstrap.cc:492] T baa7de5581644fa4a8f9fa2a721562eb P 6ccc954b995c49d893b9a21ea499404a: No bootstrap required, opened a new log
I20260806 18:33:51.062917 4449 tablet_bootstrap.cc:492] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2: No bootstrap required, opened a new log
I20260806 18:33:51.063112 4448 ts_tablet_manager.cc:1397] T baa7de5581644fa4a8f9fa2a721562eb P 6ccc954b995c49d893b9a21ea499404a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:33:51.063112 4449 ts_tablet_manager.cc:1397] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:33:51.065757 4446 raft_consensus.cc:348] T baa7de5581644fa4a8f9fa2a721562eb P 2fd60413838a4b549e10db26757efa7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd60413838a4b549e10db26757efa7b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 44361 } } peers { permanent_uuid: "328962dc02db47a38c18ae7976104fb2" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37881 } } peers { permanent_uuid: "6ccc954b995c49d893b9a21ea499404a" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45601 } }
I20260806 18:33:51.065994 4446 raft_consensus.cc:374] T baa7de5581644fa4a8f9fa2a721562eb P 2fd60413838a4b549e10db26757efa7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:51.066048 4446 raft_consensus.cc:729] T baa7de5581644fa4a8f9fa2a721562eb P 2fd60413838a4b549e10db26757efa7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd60413838a4b549e10db26757efa7b, State: Initialized, Role: FOLLOWER
I20260806 18:33:51.066540 4446 consensus_queue.cc:260] T baa7de5581644fa4a8f9fa2a721562eb P 2fd60413838a4b549e10db26757efa7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd60413838a4b549e10db26757efa7b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 44361 } } peers { permanent_uuid: "328962dc02db47a38c18ae7976104fb2" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37881 } } peers { permanent_uuid: "6ccc954b995c49d893b9a21ea499404a" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45601 } }
I20260806 18:33:51.067265 4056 heartbeater.cc:503] Master 127.7.67.254:45415 was elected leader, sending a full tablet report...
I20260806 18:33:51.067377 4448 raft_consensus.cc:348] T baa7de5581644fa4a8f9fa2a721562eb P 6ccc954b995c49d893b9a21ea499404a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd60413838a4b549e10db26757efa7b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 44361 } } peers { permanent_uuid: "328962dc02db47a38c18ae7976104fb2" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37881 } } peers { permanent_uuid: "6ccc954b995c49d893b9a21ea499404a" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45601 } }
I20260806 18:33:51.067374 4449 raft_consensus.cc:348] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd60413838a4b549e10db26757efa7b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 44361 } } peers { permanent_uuid: "328962dc02db47a38c18ae7976104fb2" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37881 } } peers { permanent_uuid: "6ccc954b995c49d893b9a21ea499404a" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45601 } }
I20260806 18:33:51.067641 4446 ts_tablet_manager.cc:1428] T baa7de5581644fa4a8f9fa2a721562eb P 2fd60413838a4b549e10db26757efa7b: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260806 18:33:51.067646 4448 raft_consensus.cc:374] T baa7de5581644fa4a8f9fa2a721562eb P 6ccc954b995c49d893b9a21ea499404a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:51.067646 4449 raft_consensus.cc:374] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:51.067699 4448 raft_consensus.cc:729] T baa7de5581644fa4a8f9fa2a721562eb P 6ccc954b995c49d893b9a21ea499404a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ccc954b995c49d893b9a21ea499404a, State: Initialized, Role: FOLLOWER
I20260806 18:33:51.067711 4449 raft_consensus.cc:729] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 328962dc02db47a38c18ae7976104fb2, State: Initialized, Role: FOLLOWER
I20260806 18:33:51.068176 4449 consensus_queue.cc:260] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd60413838a4b549e10db26757efa7b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 44361 } } peers { permanent_uuid: "328962dc02db47a38c18ae7976104fb2" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37881 } } peers { permanent_uuid: "6ccc954b995c49d893b9a21ea499404a" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45601 } }
I20260806 18:33:51.068176 4448 consensus_queue.cc:260] T baa7de5581644fa4a8f9fa2a721562eb P 6ccc954b995c49d893b9a21ea499404a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd60413838a4b549e10db26757efa7b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 44361 } } peers { permanent_uuid: "328962dc02db47a38c18ae7976104fb2" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37881 } } peers { permanent_uuid: "6ccc954b995c49d893b9a21ea499404a" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45601 } }
I20260806 18:33:51.069113 4424 heartbeater.cc:503] Master 127.7.67.254:45415 was elected leader, sending a full tablet report...
I20260806 18:33:51.069176 4449 ts_tablet_manager.cc:1428] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:33:51.069175 4448 ts_tablet_manager.cc:1428] T baa7de5581644fa4a8f9fa2a721562eb P 6ccc954b995c49d893b9a21ea499404a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:33:51.069494 4243 heartbeater.cc:503] Master 127.7.67.254:45415 was elected leader, sending a full tablet report...
I20260806 18:33:51.098619 4459 raft_consensus.cc:482] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:33:51.098838 4459 raft_consensus.cc:504] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd60413838a4b549e10db26757efa7b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 44361 } } peers { permanent_uuid: "328962dc02db47a38c18ae7976104fb2" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37881 } } peers { permanent_uuid: "6ccc954b995c49d893b9a21ea499404a" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45601 } }
I20260806 18:33:51.100023 4459 leader_election.cc:288] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2fd60413838a4b549e10db26757efa7b (127.7.67.193:44361), 6ccc954b995c49d893b9a21ea499404a (127.7.67.194:45601)
I20260806 18:33:51.103854 4442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:51.100385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:60369 (local address 127.7.67.193:44361)
0806 18:33:51.100504 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:51.100509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:51.100524 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:33:51.100603 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:51.100607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:51.100659 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:51.100733 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:51.100996 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.101656 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.102936 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.103330 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.103437 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:51.103489 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:51.103663 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 18:33:51.103730 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:33:51.103854 4428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:51.100254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:44361 (local address 127.7.67.195:60369)
0806 18:33:51.100381 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:51.100407 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:51.100510 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:51.100768 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:51.100770 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:51.100793 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:51.100898 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:51.100904 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.101748 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.101752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:51.102786 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:51.102796 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.103448 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.103452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:51.103553 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:51.103575 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:33:51.103638 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 18:33:51.103712 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":19}
I20260806 18:33:51.104012 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:51.100383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.195:60173 (local address 127.7.67.194:45601)
0806 18:33:51.100519 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:51.100523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:51.100906 (+ 383us) server_negotiation.cc:407] Connection header received
0806 18:33:51.101038 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:51.101041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:51.101096 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:51.101152 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:33:51.101406 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.101948 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.103105 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.103482 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.103592 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:51.103640 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:51.103809 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 18:33:51.103882 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:33:51.104038 4463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:51.100254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:45601 (local address 127.7.67.195:60173)
0806 18:33:51.100811 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:51.100825 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:33:51.100916 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:33:51.101181 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:33:51.101183 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:33:51.101201 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:33:51.101317 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:51.101323 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.102053 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.102056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:51.102983 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:33:51.102994 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.103597 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.103600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:33:51.103696 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:51.103715 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:33:51.103779 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 18:33:51.103847 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":205,"thread_start_us":138,"threads_started":1}
I20260806 18:33:51.104691 4173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baa7de5581644fa4a8f9fa2a721562eb" candidate_uuid: "328962dc02db47a38c18ae7976104fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ccc954b995c49d893b9a21ea499404a" is_pre_election: true
I20260806 18:33:51.104692 3995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baa7de5581644fa4a8f9fa2a721562eb" candidate_uuid: "328962dc02db47a38c18ae7976104fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd60413838a4b549e10db26757efa7b" is_pre_election: true
I20260806 18:33:51.104977 4173 raft_consensus.cc:2454] T baa7de5581644fa4a8f9fa2a721562eb P 6ccc954b995c49d893b9a21ea499404a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 328962dc02db47a38c18ae7976104fb2 in term 0.
I20260806 18:33:51.104986 3995 raft_consensus.cc:2454] T baa7de5581644fa4a8f9fa2a721562eb P 2fd60413838a4b549e10db26757efa7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 328962dc02db47a38c18ae7976104fb2 in term 0.
I20260806 18:33:51.105434 4270 leader_election.cc:302] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 328962dc02db47a38c18ae7976104fb2, 6ccc954b995c49d893b9a21ea499404a; no voters:
I20260806 18:33:51.105690 4459 raft_consensus.cc:2781] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:33:51.105775 4459 raft_consensus.cc:482] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:33:51.105825 4459 raft_consensus.cc:3037] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:51.107152 4459 raft_consensus.cc:504] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd60413838a4b549e10db26757efa7b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 44361 } } peers { permanent_uuid: "328962dc02db47a38c18ae7976104fb2" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37881 } } peers { permanent_uuid: "6ccc954b995c49d893b9a21ea499404a" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45601 } }
I20260806 18:33:51.107504 4459 leader_election.cc:288] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [CANDIDATE]: Term 1 election: Requested vote from peers 2fd60413838a4b549e10db26757efa7b (127.7.67.193:44361), 6ccc954b995c49d893b9a21ea499404a (127.7.67.194:45601)
I20260806 18:33:51.107889 4173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baa7de5581644fa4a8f9fa2a721562eb" candidate_uuid: "328962dc02db47a38c18ae7976104fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ccc954b995c49d893b9a21ea499404a"
I20260806 18:33:51.108043 3995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baa7de5581644fa4a8f9fa2a721562eb" candidate_uuid: "328962dc02db47a38c18ae7976104fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd60413838a4b549e10db26757efa7b"
I20260806 18:33:51.108026 4173 raft_consensus.cc:3037] T baa7de5581644fa4a8f9fa2a721562eb P 6ccc954b995c49d893b9a21ea499404a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:51.108165 3995 raft_consensus.cc:3037] T baa7de5581644fa4a8f9fa2a721562eb P 2fd60413838a4b549e10db26757efa7b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:51.109364 3995 raft_consensus.cc:2454] T baa7de5581644fa4a8f9fa2a721562eb P 2fd60413838a4b549e10db26757efa7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 328962dc02db47a38c18ae7976104fb2 in term 1.
I20260806 18:33:51.109364 4173 raft_consensus.cc:2454] T baa7de5581644fa4a8f9fa2a721562eb P 6ccc954b995c49d893b9a21ea499404a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 328962dc02db47a38c18ae7976104fb2 in term 1.
I20260806 18:33:51.109690 4267 leader_election.cc:302] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2fd60413838a4b549e10db26757efa7b, 328962dc02db47a38c18ae7976104fb2, 6ccc954b995c49d893b9a21ea499404a; no voters:
I20260806 18:33:51.109864 4459 raft_consensus.cc:2781] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:33:51.110119 4459 raft_consensus.cc:686] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [term 1 LEADER]: Becoming Leader. State: Replica: 328962dc02db47a38c18ae7976104fb2, State: Running, Role: LEADER
I20260806 18:33:51.110471 4459 consensus_queue.cc:237] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd60413838a4b549e10db26757efa7b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 44361 } } peers { permanent_uuid: "328962dc02db47a38c18ae7976104fb2" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37881 } } peers { permanent_uuid: "6ccc954b995c49d893b9a21ea499404a" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45601 } }
I20260806 18:33:51.113519 3788 catalog_manager.cc:5495] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 328962dc02db47a38c18ae7976104fb2 (127.7.67.195). New cstate: current_term: 1 leader_uuid: "328962dc02db47a38c18ae7976104fb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd60413838a4b549e10db26757efa7b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "328962dc02db47a38c18ae7976104fb2" member_type: VOTER last_known_addr { host: "127.7.67.195" port: 37881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ccc954b995c49d893b9a21ea499404a" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45601 } health_report { overall_health: UNKNOWN } } }
W20260806 18:33:51.114689 4057 tablet.cc:2369] T baa7de5581644fa4a8f9fa2a721562eb P 2fd60413838a4b549e10db26757efa7b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:33:51.125746 4445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:51.122262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40606 (local address 127.7.67.195:37881)
0806 18:33:51.122386 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:51.122390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:51.122405 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:33:51.122496 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:51.122499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:51.122554 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:51.122646 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:33:51.122902 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.123416 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.124269 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:51.124507 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:51.124596 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:51.124901 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:33:51.125340 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:33:51.125451 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:51.125520 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 18:33:51.125577 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:33:51.131744 3995 raft_consensus.cc:1261] T baa7de5581644fa4a8f9fa2a721562eb P 2fd60413838a4b549e10db26757efa7b [term 1 FOLLOWER]: Refusing update from remote peer 328962dc02db47a38c18ae7976104fb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:33:51.131744 4173 raft_consensus.cc:1261] T baa7de5581644fa4a8f9fa2a721562eb P 6ccc954b995c49d893b9a21ea499404a [term 1 FOLLOWER]: Refusing update from remote peer 328962dc02db47a38c18ae7976104fb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:33:51.132539 4466 consensus_queue.cc:1035] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd60413838a4b549e10db26757efa7b" member_type: VOTER last_known_addr { host: "127.7.67.193" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:33:51.132723 4459 consensus_queue.cc:1035] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ccc954b995c49d893b9a21ea499404a" member_type: VOTER last_known_addr { host: "127.7.67.194" port: 45601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:33:51.141227 4473 mvcc.cc:205] Tried to move back new op lower bound from 7315624882707787776 to 7315624882631294976. Current Snapshot: MvccSnapshot[applied={T|T < 7315624882707787776}]
I20260806 18:33:51.141884 4470 mvcc.cc:205] Tried to move back new op lower bound from 7315624882707787776 to 7315624882631294976. Current Snapshot: MvccSnapshot[applied={T|T < 7315624882707787776}]
I20260806 18:33:51.143571 4469 mvcc.cc:205] Tried to move back new op lower bound from 7315624882707787776 to 7315624882631294976. Current Snapshot: MvccSnapshot[applied={T|T < 7315624882707787776}]
I20260806 18:33:51.151237 3787 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:33:51.151400 3787 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:33:51.153764 3787 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19c38a89ce3e4b83a5e5ac585ca2a7dd: Sending DeleteTablet for 3 replicas of tablet baa7de5581644fa4a8f9fa2a721562eb
I20260806 18:33:51.154645 4148 tablet_service.cc:1514] Processing DeleteTablet for tablet baa7de5581644fa4a8f9fa2a721562eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:51 UTC) from {username='slave'} at 127.0.0.1:60388
I20260806 18:33:51.154656 3969 tablet_service.cc:1514] Processing DeleteTablet for tablet baa7de5581644fa4a8f9fa2a721562eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:51 UTC) from {username='slave'} at 127.0.0.1:38454
I20260806 18:33:51.154727 4330 tablet_service.cc:1514] Processing DeleteTablet for tablet baa7de5581644fa4a8f9fa2a721562eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:33:51 UTC) from {username='slave'} at 127.0.0.1:40600
I20260806 18:33:51.155320 4487 tablet_replica.cc:330] T baa7de5581644fa4a8f9fa2a721562eb P 2fd60413838a4b549e10db26757efa7b: stopping tablet replica
I20260806 18:33:51.155501 4485 tablet_replica.cc:330] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2: stopping tablet replica
I20260806 18:33:51.155725 4487 raft_consensus.cc:2229] T baa7de5581644fa4a8f9fa2a721562eb P 2fd60413838a4b549e10db26757efa7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:33:51.155797 4485 raft_consensus.cc:2229] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:33:51.156057 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 3881
I20260806 18:33:51.156157 4487 raft_consensus.cc:2258] T baa7de5581644fa4a8f9fa2a721562eb P 2fd60413838a4b549e10db26757efa7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:33:51.179646 4486 tablet_replica.cc:330] T baa7de5581644fa4a8f9fa2a721562eb P 6ccc954b995c49d893b9a21ea499404a: stopping tablet replica
I20260806 18:33:51.180076 4486 raft_consensus.cc:2229] T baa7de5581644fa4a8f9fa2a721562eb P 6ccc954b995c49d893b9a21ea499404a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:33:51.180490 4486 raft_consensus.cc:2258] T baa7de5581644fa4a8f9fa2a721562eb P 6ccc954b995c49d893b9a21ea499404a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:33:51.180814 4485 raft_consensus.cc:2258] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:33:51.182596 4485 ts_tablet_manager.cc:1905] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 18:33:51.185250 3770 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.193:44361 (error 108)
I20260806 18:33:51.185492 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 4060
I20260806 18:33:51.185523 4485 ts_tablet_manager.cc:1918] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:33:51.185616 4485 log.cc:1196] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2: Deleting WAL directory at /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786040973123132-7439-0/minicluster-data/ts-2/wal/wals/baa7de5581644fa4a8f9fa2a721562eb
I20260806 18:33:51.185874 4485 ts_tablet_manager.cc:1939] T baa7de5581644fa4a8f9fa2a721562eb P 328962dc02db47a38c18ae7976104fb2: Deleting consensus metadata
I20260806 18:33:51.189525 3769 catalog_manager.cc:4841] TS 328962dc02db47a38c18ae7976104fb2 (127.7.67.195:37881): tablet baa7de5581644fa4a8f9fa2a721562eb (table test-table [id=040010d3b2c24dd5aa50bc1edb89bce4]) successfully deleted
I20260806 18:33:51.189985 4441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:51.189371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.193:44361 (local address 127.0.0.1:38466)
0806 18:33:51.189813 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:51.189891 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.193:44361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354}
W20260806 18:33:51.190186 3770 catalog_manager.cc:4584] TS 2fd60413838a4b549e10db26757efa7b (127.7.67.193:44361): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet baa7de5581644fa4a8f9fa2a721562eb: Network error: Client connection negotiation failed: client connection to 127.7.67.193:44361: connect: Connection refused (error 111)
I20260806 18:33:51.198729 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 4248
I20260806 18:33:51.206586 4441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:51.198637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.194:45601 (local address 127.0.0.1:60390)
0806 18:33:51.198766 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 18:33:51.198827 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.194:45601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 18:33:51.212121 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 3750
2026-08-06T18:33:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1070 ms)
[----------] 6 tests from EncryptionPolicyTest (11016 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:33:51 dist-test-slave-5bn2 krb5kdc[4495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:33:51 dist-test-slave-5bn2 krb5kdc[4495](info): set up 2 sockets
Aug 06 18:33:51 dist-test-slave-5bn2 krb5kdc[4495](info): commencing operation
krb5kdc: starting...
W20260806 18:33:52.339581 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:33:52 dist-test-slave-5bn2 krb5kdc[4495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:33:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:33:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:33:52.465610 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45023
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:33:52.580183 4514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:52.580533 4514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:52.580592 4514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:52.584767 4514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:33:52.584842 4514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:52.584869 4514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:52.584890 4514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:33:52.584911 4514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:33:52.590824 4514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45023
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:52.592327 4514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:52.593353 4514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:33:52.602137 4514 server_base.cc:1034] running on GCE node
W20260806 18:33:52.602099 4523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:52.602099 4524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:52.602402 4526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:52.603011 4514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:52.604116 4514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:52.605275 4514 hybrid_clock.cc:648] HybridClock initialized: now 1786041232605260 us; error 27 us; skew 500 ppm
Aug 06 18:33:52 dist-test-slave-5bn2 krb5kdc[4495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041232, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:52.607961 4514 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:33:52.608857 4514 webserver.cc:460] Webserver started at http://127.0.0.1:37573/ using document root <none> and password file <none>
I20260806 18:33:52.609412 4514 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:52.609481 4514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:52.609678 4514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:52.611811 4514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "efa8483151914da4a993cf25f31f83be"
format_stamp: "Formatted at 2026-08-06 18:33:52 on dist-test-slave-5bn2"
server_key: "fea0721159ff33eb5b93cafacb0b69a1"
server_key_iv: "87ab97453f0a28a06dac74f9f6fac6a4"
server_key_version: "encryptionkey@0"
I20260806 18:33:52.612377 4514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "efa8483151914da4a993cf25f31f83be"
format_stamp: "Formatted at 2026-08-06 18:33:52 on dist-test-slave-5bn2"
server_key: "fea0721159ff33eb5b93cafacb0b69a1"
server_key_iv: "87ab97453f0a28a06dac74f9f6fac6a4"
server_key_version: "encryptionkey@0"
I20260806 18:33:52.616134 4514 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:33:52.618396 4535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:52.619597 4514 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:33:52.619716 4514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "efa8483151914da4a993cf25f31f83be"
format_stamp: "Formatted at 2026-08-06 18:33:52 on dist-test-slave-5bn2"
server_key: "fea0721159ff33eb5b93cafacb0b69a1"
server_key_iv: "87ab97453f0a28a06dac74f9f6fac6a4"
server_key_version: "encryptionkey@0"
I20260806 18:33:52.619813 4514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:52.643146 4514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:52.646013 4514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:52.646212 4514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:52.653355 4514 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45023
I20260806 18:33:52.653366 4610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45023 every 8 connection(s)
I20260806 18:33:52.654624 4514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:33:52.657420 4612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:52.663008 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 4514
I20260806 18:33:52.663159 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:33:52.663169 4612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be: Bootstrap starting.
I20260806 18:33:52.663501 7439 external_mini_cluster.cc:1428] Setting key d48a583b73d519c171b9e0d0e121438b
I20260806 18:33:52.665850 4612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:52.666582 4612 log.cc:824] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:52.668489 4612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be: No bootstrap required, opened a new log
Aug 06 18:33:52 dist-test-slave-5bn2 krb5kdc[4495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:33:52.672147 4612 raft_consensus.cc:348] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa8483151914da4a993cf25f31f83be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45023 } }
I20260806 18:33:52.672433 4612 raft_consensus.cc:374] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:52.672497 4612 raft_consensus.cc:729] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa8483151914da4a993cf25f31f83be, State: Initialized, Role: FOLLOWER
I20260806 18:33:52.673007 4612 consensus_queue.cc:260] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa8483151914da4a993cf25f31f83be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45023 } }
I20260806 18:33:52.673148 4612 raft_consensus.cc:388] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:33:52.673207 4612 raft_consensus.cc:482] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:33:52.673305 4612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:52.674199 4612 raft_consensus.cc:504] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa8483151914da4a993cf25f31f83be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45023 } }
I20260806 18:33:52.674535 4612 leader_election.cc:302] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efa8483151914da4a993cf25f31f83be; no voters:
I20260806 18:33:52.674870 4612 leader_election.cc:288] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:33:52.674986 4619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:33:52.675163 4619 raft_consensus.cc:686] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be [term 1 LEADER]: Becoming Leader. State: Replica: efa8483151914da4a993cf25f31f83be, State: Running, Role: LEADER
I20260806 18:33:52.675478 4619 consensus_queue.cc:237] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa8483151914da4a993cf25f31f83be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45023 } }
I20260806 18:33:52.676263 4612 sys_catalog.cc:564] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:33:52.677963 4622 sys_catalog.cc:455] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efa8483151914da4a993cf25f31f83be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa8483151914da4a993cf25f31f83be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45023 } } }
I20260806 18:33:52.678107 4622 sys_catalog.cc:458] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:52.678362 4624 sys_catalog.cc:455] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be [sys.catalog]: SysCatalogTable state changed. Reason: New leader efa8483151914da4a993cf25f31f83be. Latest consensus state: current_term: 1 leader_uuid: "efa8483151914da4a993cf25f31f83be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa8483151914da4a993cf25f31f83be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45023 } } }
I20260806 18:33:52.678375 4616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:52.664967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37959 (local address 127.0.0.1:45023)
0806 18:33:52.665411 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:52.665427 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:52.665465 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:33:52.666158 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:52.666178 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:52.666531 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:52.666887 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:52.667712 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:52.668736 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:52.669581 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:52.670101 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:52.670250 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:52.672983 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:52.672989 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:52.673002 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:52.673032 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:52.674811 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:52.675501 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:52.675508 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:52.675514 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:52.675671 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:52.676025 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:52.676029 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:52.676031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:52.676565 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:52.676797 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:52.677142 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 18:33:52.677381 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":126,"threads_started":1}
I20260806 18:33:52.678462 4624 sys_catalog.cc:458] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:52.679031 4642 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:33:52.681717 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:33:52.682313 4642 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:33:52.688315 4642 catalog_manager.cc:1347] Generated new cluster ID: 610b882ea1874cfd9849d1d773ff5b43
I20260806 18:33:52.688386 4642 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:33:52.692062 4616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:52.684053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52597 (local address 127.0.0.1:45023)
0806 18:33:52.684156 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:52.684160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:52.684483 (+ 323us) server_negotiation.cc:407] Connection header received
0806 18:33:52.684586 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:52.684590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:52.684633 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:52.684694 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:52.685372 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:52.685941 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:52.686993 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:52.687358 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:52.687471 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:52.688980 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:52.688984 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:52.688986 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:52.689011 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:52.690344 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:52.690796 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:52.690799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:52.690800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:52.690901 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:52.691225 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:52.691228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:52.691230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:52.691401 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:52.691535 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:52.691799 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 18:33:52.691892 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 18:33:52.710289 4642 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:33:52.711323 4642 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:33:52.721836 4642 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efa8483151914da4a993cf25f31f83be: Generated new TSK 0
I20260806 18:33:52.722543 4642 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:33:52.735244 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 4514
W20260806 18:33:52.747480 4491 connection.cc:535] client connection to 127.0.0.1:45023 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45023): Connection reset by peer (error 104)
2026-08-06T18:33:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1524 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:33:52 dist-test-slave-5bn2 krb5kdc[4659](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:33:52 dist-test-slave-5bn2 krb5kdc[4659](info): set up 2 sockets
Aug 06 18:33:52 dist-test-slave-5bn2 krb5kdc[4659](info): commencing operation
krb5kdc: starting...
W20260806 18:33:53.860419 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:33:53 dist-test-slave-5bn2 krb5kdc[4659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:33:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:33:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:33:53.989414 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40239
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:33:54.104342 4676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:54.104619 4676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:54.104665 4676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:54.108703 4676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:33:54.108772 4676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:54.108793 4676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:54.108821 4676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:33:54.108848 4676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:33:54.114154 4676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40239
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:54.115233 4676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:54.116290 4676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:54.124581 4689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:54.124804 4684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:54.124889 4686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:54.124960 4676 server_base.cc:1034] running on GCE node
I20260806 18:33:54.125512 4676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:54.126608 4676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:54.127763 4676 hybrid_clock.cc:648] HybridClock initialized: now 1786041234127744 us; error 31 us; skew 500 ppm
Aug 06 18:33:54 dist-test-slave-5bn2 krb5kdc[4659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041234, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:54.133212 4676 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:33:54.134125 4676 webserver.cc:460] Webserver started at http://127.0.0.1:45685/ using document root <none> and password file <none>
I20260806 18:33:54.134675 4676 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:54.134737 4676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:54.134976 4676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:54.137038 4676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "144f8dc2ef0e47fda5f811aa79ded0df"
format_stamp: "Formatted at 2026-08-06 18:33:54 on dist-test-slave-5bn2"
server_key: "267ef55f18abeab87f2c840e1fe7d7d1"
server_key_iv: "4c4c5d6809ed57cc6e4546bde23aba7b"
server_key_version: "encryptionkey@0"
I20260806 18:33:54.137603 4676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "144f8dc2ef0e47fda5f811aa79ded0df"
format_stamp: "Formatted at 2026-08-06 18:33:54 on dist-test-slave-5bn2"
server_key: "267ef55f18abeab87f2c840e1fe7d7d1"
server_key_iv: "4c4c5d6809ed57cc6e4546bde23aba7b"
server_key_version: "encryptionkey@0"
I20260806 18:33:54.141363 4676 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 18:33:54.143849 4700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:54.145066 4676 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:33:54.145179 4676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "144f8dc2ef0e47fda5f811aa79ded0df"
format_stamp: "Formatted at 2026-08-06 18:33:54 on dist-test-slave-5bn2"
server_key: "267ef55f18abeab87f2c840e1fe7d7d1"
server_key_iv: "4c4c5d6809ed57cc6e4546bde23aba7b"
server_key_version: "encryptionkey@0"
I20260806 18:33:54.145287 4676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:54.168814 4676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:54.171989 4676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:54.172163 4676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:54.179005 4676 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40239
I20260806 18:33:54.179044 4786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40239 every 8 connection(s)
I20260806 18:33:54.179965 4676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:33:54.182750 4787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:54.186697 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 4676
I20260806 18:33:54.186892 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:33:54.187234 7439 external_mini_cluster.cc:1428] Setting key 0c54df753281c0925506ae2435cdfdfb
I20260806 18:33:54.189136 4787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df: Bootstrap starting.
I20260806 18:33:54.191560 4787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:54.192324 4787 log.cc:824] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:54.194259 4787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df: No bootstrap required, opened a new log
I20260806 18:33:54.196448 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:54.188770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39005 (local address 127.0.0.1:40239)
0806 18:33:54.189222 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:54.189242 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:54.189288 (+ 46us) server_negotiation.cc:407] Connection header received
0806 18:33:54.189950 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:54.189968 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:54.190347 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:54.190660 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:54.191126 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:54.192180 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:54.193087 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:54.193557 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:54.193702 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:54.194551 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:54.194556 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:54.194570 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:54.194999 (+ 429us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:54.195065 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:54.195089 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:54.195200 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:54.195388 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 18:33:54.195523 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":127,"threads_started":1}
I20260806 18:33:54.197435 4787 raft_consensus.cc:348] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144f8dc2ef0e47fda5f811aa79ded0df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40239 } }
I20260806 18:33:54.197669 4787 raft_consensus.cc:374] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:54.197728 4787 raft_consensus.cc:729] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144f8dc2ef0e47fda5f811aa79ded0df, State: Initialized, Role: FOLLOWER
I20260806 18:33:54.198192 4787 consensus_queue.cc:260] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144f8dc2ef0e47fda5f811aa79ded0df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40239 } }
I20260806 18:33:54.198357 4787 raft_consensus.cc:388] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:33:54.198426 4787 raft_consensus.cc:482] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:33:54.198525 4787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:54.199362 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:33:54.199597 4787 raft_consensus.cc:504] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144f8dc2ef0e47fda5f811aa79ded0df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40239 } }
I20260806 18:33:54.199973 4787 leader_election.cc:302] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 144f8dc2ef0e47fda5f811aa79ded0df; no voters:
I20260806 18:33:54.200388 4787 leader_election.cc:288] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:33:54.200469 4795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:33:54.200723 4795 raft_consensus.cc:686] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df [term 1 LEADER]: Becoming Leader. State: Replica: 144f8dc2ef0e47fda5f811aa79ded0df, State: Running, Role: LEADER
I20260806 18:33:54.201027 4795 consensus_queue.cc:237] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144f8dc2ef0e47fda5f811aa79ded0df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40239 } }
I20260806 18:33:54.201774 4787 sys_catalog.cc:564] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:33:54.202129 4795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 144f8dc2ef0e47fda5f811aa79ded0df. Latest consensus state: current_term: 1 leader_uuid: "144f8dc2ef0e47fda5f811aa79ded0df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144f8dc2ef0e47fda5f811aa79ded0df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40239 } } }
I20260806 18:33:54.202247 4795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:54.202647 4797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "144f8dc2ef0e47fda5f811aa79ded0df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144f8dc2ef0e47fda5f811aa79ded0df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40239 } } }
I20260806 18:33:54.202751 4797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:54.203277 4807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:33:54.206504 4807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:33:54.207815 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:54.202778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41441 (local address 127.0.0.1:40239)
0806 18:33:54.202943 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:54.202947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:54.203160 (+ 213us) server_negotiation.cc:407] Connection header received
0806 18:33:54.203357 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:54.203360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:54.203415 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:54.203469 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:54.203955 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:54.204737 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:54.205931 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:54.206185 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:54.206277 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:54.207279 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:54.207283 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:54.207287 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:54.207302 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:54.207334 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:54.207348 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:54.207411 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:54.207642 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 18:33:54.207679 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 18:33:54.212785 4807 catalog_manager.cc:1347] Generated new cluster ID: ca82bfa89f1f4a84ac9060fc8e3fbbcc
I20260806 18:33:54.212864 4807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:33:54.229324 4807 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:33:54.230616 4807 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:33:54.244935 4807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 144f8dc2ef0e47fda5f811aa79ded0df: Generated new TSK 0
I20260806 18:33:54.245667 4807 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:33:54.255587 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 4676
W20260806 18:33:54.271910 4656 connection.cc:535] client connection to 127.0.0.1:40239 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40239): Connection reset by peer (error 104)
2026-08-06T18:33:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1524 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:33:54 dist-test-slave-5bn2 krb5kdc[4832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:33:54 dist-test-slave-5bn2 krb5kdc[4832](info): set up 2 sockets
Aug 06 18:33:54 dist-test-slave-5bn2 krb5kdc[4832](info): commencing operation
krb5kdc: starting...
W20260806 18:33:55.386996 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:33:55 dist-test-slave-5bn2 krb5kdc[4832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:33:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:33:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:33:55.511590 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43961
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:33:55.625027 4849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:55.625305 4849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:55.625346 4849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:55.629335 4849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:33:55.629400 4849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:55.629418 4849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:55.629434 4849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:33:55.629447 4849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:33:55.634718 4849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43961
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:55.635807 4849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:55.636785 4849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:55.645206 4855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:55.645201 4859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:55.645274 4857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:55.645642 4849 server_base.cc:1034] running on GCE node
I20260806 18:33:55.646202 4849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:55.647377 4849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:55.648562 4849 hybrid_clock.cc:648] HybridClock initialized: now 1786041235648547 us; error 27 us; skew 500 ppm
Aug 06 18:33:55 dist-test-slave-5bn2 krb5kdc[4832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041235, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:55.651393 4849 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:33:55.652344 4849 webserver.cc:460] Webserver started at http://127.0.0.1:37865/ using document root <none> and password file <none>
I20260806 18:33:55.652908 4849 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:55.652980 4849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:55.653180 4849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:55.655328 4849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "4a45799a1af24e5eb1949f936a50b18c"
format_stamp: "Formatted at 2026-08-06 18:33:55 on dist-test-slave-5bn2"
server_key: "c21ea8597975aae884f8b7579b130595"
server_key_iv: "e718084ff6bf45b017d59eefc87ad0bf"
server_key_version: "encryptionkey@0"
I20260806 18:33:55.655894 4849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "4a45799a1af24e5eb1949f936a50b18c"
format_stamp: "Formatted at 2026-08-06 18:33:55 on dist-test-slave-5bn2"
server_key: "c21ea8597975aae884f8b7579b130595"
server_key_iv: "e718084ff6bf45b017d59eefc87ad0bf"
server_key_version: "encryptionkey@0"
I20260806 18:33:55.659647 4849 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:33:55.662041 4868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:55.663282 4849 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:33:55.663399 4849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "4a45799a1af24e5eb1949f936a50b18c"
format_stamp: "Formatted at 2026-08-06 18:33:55 on dist-test-slave-5bn2"
server_key: "c21ea8597975aae884f8b7579b130595"
server_key_iv: "e718084ff6bf45b017d59eefc87ad0bf"
server_key_version: "encryptionkey@0"
I20260806 18:33:55.663515 4849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:55.668097 4849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:55.671216 4849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:55.671386 4849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:55.678365 4849 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43961
I20260806 18:33:55.678375 4946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43961 every 8 connection(s)
I20260806 18:33:55.679345 4849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:33:55.682132 4947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:55.687651 4947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c: Bootstrap starting.
I20260806 18:33:55.688213 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 4849
I20260806 18:33:55.688364 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:33:55.688660 7439 external_mini_cluster.cc:1428] Setting key e8348273535f80c2aed29d7db1392fbf
I20260806 18:33:55.690548 4947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:55.691291 4947 log.cc:824] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:55.693205 4947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c: No bootstrap required, opened a new log
I20260806 18:33:55.693852 4952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:55.690004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33175 (local address 127.0.0.1:43961)
0806 18:33:55.690485 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:55.690499 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:55.690535 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:33:55.691236 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:55.691255 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:55.691574 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:55.691875 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:55.692662 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:55.692667 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:55.692678 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:55.692689 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:55.692745 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:55.692764 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:55.692859 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:55.692998 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 18:33:55.693150 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":131,"threads_started":1}
I20260806 18:33:55.696530 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:33:55.696852 4947 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a45799a1af24e5eb1949f936a50b18c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43961 } }
I20260806 18:33:55.697064 4947 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:55.697116 4947 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a45799a1af24e5eb1949f936a50b18c, State: Initialized, Role: FOLLOWER
I20260806 18:33:55.697526 4947 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a45799a1af24e5eb1949f936a50b18c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43961 } }
I20260806 18:33:55.697669 4947 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:33:55.697739 4947 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:33:55.697885 4947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:55.699033 4947 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a45799a1af24e5eb1949f936a50b18c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43961 } }
I20260806 18:33:55.699407 4947 leader_election.cc:302] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a45799a1af24e5eb1949f936a50b18c; no voters:
I20260806 18:33:55.699759 4947 leader_election.cc:288] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:33:55.699843 4960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:33:55.700070 4960 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c [term 1 LEADER]: Becoming Leader. State: Replica: 4a45799a1af24e5eb1949f936a50b18c, State: Running, Role: LEADER
I20260806 18:33:55.700417 4952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:55.698516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40009 (local address 127.0.0.1:43961)
0806 18:33:55.698616 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:55.698619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:55.698845 (+ 226us) server_negotiation.cc:407] Connection header received
0806 18:33:55.699006 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:55.699008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:55.699039 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:55.699101 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:55.699824 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:55.699828 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:55.699831 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:55.699844 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:55.699872 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:55.699885 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:55.699979 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:55.700269 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 18:33:55.700306 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 18:33:55.700469 4960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a45799a1af24e5eb1949f936a50b18c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43961 } }
I20260806 18:33:55.701004 4947 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:33:55.702666 4962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a45799a1af24e5eb1949f936a50b18c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a45799a1af24e5eb1949f936a50b18c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43961 } } }
I20260806 18:33:55.702664 4965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a45799a1af24e5eb1949f936a50b18c. Latest consensus state: current_term: 1 leader_uuid: "4a45799a1af24e5eb1949f936a50b18c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a45799a1af24e5eb1949f936a50b18c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43961 } } }
I20260806 18:33:55.702818 4962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:55.702818 4965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:55.703245 4984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:33:55.706120 4984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:33:55.712076 4984 catalog_manager.cc:1347] Generated new cluster ID: 4a3f4979dd6b4421826306384118e7b3
I20260806 18:33:55.712150 4984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:33:55.734234 4984 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:33:55.734557 4984 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:33:55.741122 4984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a45799a1af24e5eb1949f936a50b18c: Generated new TSK 0
I20260806 18:33:55.741745 4984 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:33:55.745808 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 4849
2026-08-06T18:33:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1485 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:33:55 dist-test-slave-5bn2 krb5kdc[4997](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:33:55 dist-test-slave-5bn2 krb5kdc[4997](info): set up 2 sockets
Aug 06 18:33:55 dist-test-slave-5bn2 krb5kdc[4997](info): commencing operation
krb5kdc: starting...
W20260806 18:33:56.866606 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:33:56 dist-test-slave-5bn2 krb5kdc[4997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:33:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:33:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:33:56.997694 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40833
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:33:57.113050 5012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:57.113353 5012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:57.113425 5012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:57.117518 5012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:33:57.117597 5012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:57.117624 5012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:57.117645 5012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:33:57.117664 5012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:33:57.123204 5012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:57.124382 5012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:57.125478 5012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:57.134369 5025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:57.134370 5021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:57.134370 5019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:57.134430 5012 server_base.cc:1034] running on GCE node
I20260806 18:33:57.135317 5012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:57.136502 5012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:57.137678 5012 hybrid_clock.cc:648] HybridClock initialized: now 1786041237137662 us; error 28 us; skew 500 ppm
Aug 06 18:33:57 dist-test-slave-5bn2 krb5kdc[4997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041237, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:57.140656 5012 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:33:57.141639 5012 webserver.cc:460] Webserver started at http://127.0.0.1:35517/ using document root <none> and password file <none>
I20260806 18:33:57.142184 5012 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:57.142246 5012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:57.142421 5012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:57.144613 5012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "6167701bdfc94a8399f70e0eba2aa520"
format_stamp: "Formatted at 2026-08-06 18:33:57 on dist-test-slave-5bn2"
server_key: "fb7052fd524ac277483e200cffad63e2"
server_key_iv: "e8ec3495efb59bc7d79a9d6c853439b7"
server_key_version: "encryptionkey@0"
I20260806 18:33:57.145186 5012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "6167701bdfc94a8399f70e0eba2aa520"
format_stamp: "Formatted at 2026-08-06 18:33:57 on dist-test-slave-5bn2"
server_key: "fb7052fd524ac277483e200cffad63e2"
server_key_iv: "e8ec3495efb59bc7d79a9d6c853439b7"
server_key_version: "encryptionkey@0"
I20260806 18:33:57.149121 5012 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:33:57.151696 5032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:57.152961 5012 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:33:57.153079 5012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "6167701bdfc94a8399f70e0eba2aa520"
format_stamp: "Formatted at 2026-08-06 18:33:57 on dist-test-slave-5bn2"
server_key: "fb7052fd524ac277483e200cffad63e2"
server_key_iv: "e8ec3495efb59bc7d79a9d6c853439b7"
server_key_version: "encryptionkey@0"
I20260806 18:33:57.153195 5012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:57.164788 5012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:57.168073 5012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:57.168238 5012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:57.175567 5012 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40833
I20260806 18:33:57.175623 5101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40833 every 8 connection(s)
I20260806 18:33:57.176582 5012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:33:57.179579 5102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:57.185461 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 5012
I20260806 18:33:57.185477 5102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520: Bootstrap starting.
I20260806 18:33:57.185587 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:33:57.185842 7439 external_mini_cluster.cc:1428] Setting key d15a78d77860e85d62140a26d58749c8
I20260806 18:33:57.188318 5102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:57.189278 5102 log.cc:824] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:57.191726 5102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520: No bootstrap required, opened a new log
Aug 06 18:33:57 dist-test-slave-5bn2 krb5kdc[4997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:33:57.195778 5102 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6167701bdfc94a8399f70e0eba2aa520" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40833 } }
I20260806 18:33:57.196075 5102 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:57.196137 5102 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6167701bdfc94a8399f70e0eba2aa520, State: Initialized, Role: FOLLOWER
I20260806 18:33:57.196647 5102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6167701bdfc94a8399f70e0eba2aa520" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40833 } }
I20260806 18:33:57.196820 5102 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:33:57.196893 5102 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:33:57.196986 5102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:57.198246 5102 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6167701bdfc94a8399f70e0eba2aa520" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40833 } }
I20260806 18:33:57.198650 5102 leader_election.cc:302] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6167701bdfc94a8399f70e0eba2aa520; no voters:
I20260806 18:33:57.199012 5102 leader_election.cc:288] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:33:57.199119 5107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:33:57.199339 5107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520 [term 1 LEADER]: Becoming Leader. State: Replica: 6167701bdfc94a8399f70e0eba2aa520, State: Running, Role: LEADER
I20260806 18:33:57.199697 5107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6167701bdfc94a8399f70e0eba2aa520" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40833 } }
I20260806 18:33:57.200436 5102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:33:57.201399 5110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6167701bdfc94a8399f70e0eba2aa520. Latest consensus state: current_term: 1 leader_uuid: "6167701bdfc94a8399f70e0eba2aa520" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6167701bdfc94a8399f70e0eba2aa520" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40833 } } }
I20260806 18:33:57.201525 5110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520 [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:57.201740 5105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:57.187265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48423 (local address 127.0.0.1:40833)
0806 18:33:57.187746 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:57.187765 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:57.187811 (+ 46us) server_negotiation.cc:407] Connection header received
0806 18:33:57.188683 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:57.188708 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:57.189179 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:57.189560 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:57.190423 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:57.191884 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:57.192765 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:57.193462 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:57.193625 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:57.195967 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:57.195973 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:57.195990 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:57.196024 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:57.197962 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:57.198392 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:57.198398 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:57.198405 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:57.198556 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:57.198877 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:57.198880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:57.198882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:57.199432 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:57.200254 (+ 822us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:57.200407 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 18:33:57.200714 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":131,"threads_started":1}
I20260806 18:33:57.201767 5108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6167701bdfc94a8399f70e0eba2aa520" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6167701bdfc94a8399f70e0eba2aa520" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40833 } } }
I20260806 18:33:57.201857 5108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520 [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:57.201920 5120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:33:57.205097 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:33:57.205152 5120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:33:57.211069 5120 catalog_manager.cc:1347] Generated new cluster ID: a959ecfbff0a4ef58bb99fa008591255
I20260806 18:33:57.211138 5120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:33:57.218107 5105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:57.209302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60263 (local address 127.0.0.1:40833)
0806 18:33:57.209443 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:57.209448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:57.209762 (+ 314us) server_negotiation.cc:407] Connection header received
0806 18:33:57.209866 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:57.209869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:57.209924 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:57.210005 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:57.210891 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:57.211702 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:57.212907 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:57.213258 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:57.213352 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:57.214978 (+ 1626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:57.214981 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:57.214984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:33:57.215010 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:57.216531 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:57.216986 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:57.216990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:57.216991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:57.217068 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:33:57.217314 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:33:57.217317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:33:57.217319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:33:57.217511 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:57.217621 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:57.217839 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 18:33:57.217947 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:33:57.225344 5120 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:33:57.226580 5120 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:33:57.234114 5120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6167701bdfc94a8399f70e0eba2aa520: Generated new TSK 0
I20260806 18:33:57.234956 5120 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:33:57.244302 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 5012
2026-08-06T18:33:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1497 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:33:57 dist-test-slave-5bn2 krb5kdc[5146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:33:57 dist-test-slave-5bn2 krb5kdc[5146](info): set up 2 sockets
Aug 06 18:33:57 dist-test-slave-5bn2 krb5kdc[5146](info): commencing operation
krb5kdc: starting...
W20260806 18:33:58.366156 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:33:58 dist-test-slave-5bn2 krb5kdc[5146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:33:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:33:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:33:58.494163 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40827
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:33:58.607445 5164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:33:58.607759 5164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:33:58.607822 5164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:33:58.612507 5164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:33:58.612598 5164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:33:58.612628 5164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:33:58.612651 5164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:33:58.612727 5164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:33:58.618604 5164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:33:58.619740 5164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:33:58.620769 5164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:33:58.629128 5170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:58.629165 5175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:33:58.629165 5171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:33:58.629459 5164 server_base.cc:1034] running on GCE node
I20260806 18:33:58.630028 5164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:33:58.631175 5164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:33:58.632326 5164 hybrid_clock.cc:648] HybridClock initialized: now 1786041238632307 us; error 32 us; skew 500 ppm
Aug 06 18:33:58 dist-test-slave-5bn2 krb5kdc[5146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041238, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:33:58.635066 5164 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:33:58.635986 5164 webserver.cc:460] Webserver started at http://127.0.0.1:37241/ using document root <none> and password file <none>
I20260806 18:33:58.636540 5164 fs_manager.cc:359] Metadata directory not provided
I20260806 18:33:58.636600 5164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:33:58.636792 5164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:33:58.638842 5164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "b5e9f709fce547f8a0c9b8843e827fb8"
format_stamp: "Formatted at 2026-08-06 18:33:58 on dist-test-slave-5bn2"
server_key: "638d729ae4920997f66c5e849a436c6f"
server_key_iv: "4249d394ccfcb4a22d85f364e59989f3"
server_key_version: "encryptionkey@0"
I20260806 18:33:58.639463 5164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "b5e9f709fce547f8a0c9b8843e827fb8"
format_stamp: "Formatted at 2026-08-06 18:33:58 on dist-test-slave-5bn2"
server_key: "638d729ae4920997f66c5e849a436c6f"
server_key_iv: "4249d394ccfcb4a22d85f364e59989f3"
server_key_version: "encryptionkey@0"
I20260806 18:33:58.643184 5164 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:33:58.645562 5186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:33:58.646731 5164 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:33:58.646842 5164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "b5e9f709fce547f8a0c9b8843e827fb8"
format_stamp: "Formatted at 2026-08-06 18:33:58 on dist-test-slave-5bn2"
server_key: "638d729ae4920997f66c5e849a436c6f"
server_key_iv: "4249d394ccfcb4a22d85f364e59989f3"
server_key_version: "encryptionkey@0"
I20260806 18:33:58.646996 5164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:33:58.662226 5164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:33:58.665321 5164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:33:58.665517 5164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:33:58.672962 5164 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40827
I20260806 18:33:58.672997 5254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40827 every 8 connection(s)
I20260806 18:33:58.673943 5164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:33:58.676775 5255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:33:58.681046 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 5164
I20260806 18:33:58.681186 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:33:58.681447 7439 external_mini_cluster.cc:1428] Setting key 49a758b0ceb823bddc4674aeb0694645
I20260806 18:33:58.682636 5255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8: Bootstrap starting.
I20260806 18:33:58.684921 5255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8: Neither blocks nor log segments found. Creating new log.
I20260806 18:33:58.685667 5255 log.cc:824] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8: Log is configured to *not* fsync() on all Append() calls
I20260806 18:33:58.687525 5255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8: No bootstrap required, opened a new log
I20260806 18:33:58.689989 5258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:58.682778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57501 (local address 127.0.0.1:40827)
0806 18:33:58.683240 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:58.683254 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:58.683292 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:33:58.683917 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:58.683933 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:58.684263 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:58.684578 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:58.684991 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:58.686024 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:58.686947 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:58.687406 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:58.687566 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:58.688348 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:58.688353 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:58.688367 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:58.688770 (+ 403us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:58.688823 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:58.688846 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:58.688930 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:58.689083 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 18:33:58.689285 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":132,"threads_started":1}
I20260806 18:33:58.691095 5255 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e9f709fce547f8a0c9b8843e827fb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40827 } }
I20260806 18:33:58.691323 5255 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:33:58.691380 5255 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5e9f709fce547f8a0c9b8843e827fb8, State: Initialized, Role: FOLLOWER
I20260806 18:33:58.691804 5255 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e9f709fce547f8a0c9b8843e827fb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40827 } }
I20260806 18:33:58.691947 5255 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:33:58.692014 5255 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:33:58.692117 5255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:33:58.692754 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:33:58.693105 5255 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e9f709fce547f8a0c9b8843e827fb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40827 } }
I20260806 18:33:58.693413 5255 leader_election.cc:302] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5e9f709fce547f8a0c9b8843e827fb8; no voters:
I20260806 18:33:58.693744 5255 leader_election.cc:288] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:33:58.693881 5260 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:33:58.694136 5260 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8 [term 1 LEADER]: Becoming Leader. State: Replica: b5e9f709fce547f8a0c9b8843e827fb8, State: Running, Role: LEADER
I20260806 18:33:58.694481 5260 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e9f709fce547f8a0c9b8843e827fb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40827 } }
I20260806 18:33:58.695047 5255 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:33:58.696280 5268 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5e9f709fce547f8a0c9b8843e827fb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e9f709fce547f8a0c9b8843e827fb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40827 } } }
I20260806 18:33:58.696399 5268 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8 [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:58.696676 5264 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5e9f709fce547f8a0c9b8843e827fb8. Latest consensus state: current_term: 1 leader_uuid: "b5e9f709fce547f8a0c9b8843e827fb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e9f709fce547f8a0c9b8843e827fb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40827 } } }
I20260806 18:33:58.696827 5264 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8 [sys.catalog]: This master's current role is: LEADER
I20260806 18:33:58.699023 5258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:33:58.694708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57759 (local address 127.0.0.1:40827)
0806 18:33:58.694809 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 18:33:58.694813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:33:58.695079 (+ 266us) server_negotiation.cc:407] Connection header received
0806 18:33:58.695226 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:33:58.695229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:33:58.695271 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:33:58.695335 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:33:58.695589 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:58.696304 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:58.697409 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:33:58.697719 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:33:58.697818 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:33:58.698556 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:33:58.698559 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:33:58.698562 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:33:58.698574 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:33:58.698603 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:33:58.698617 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:33:58.698696 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 18:33:58.698813 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 18:33:58.698890 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 18:33:58.700167 5287 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:33:58.700232 5287 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:33:58.700343 5281 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:33:58.701256 5281 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:33:58.706918 5281 catalog_manager.cc:1347] Generated new cluster ID: 525a04d7988345f2875463d50b3fda63
I20260806 18:33:58.706990 5281 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:33:58.715492 5281 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:33:58.716392 5281 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:33:58.723059 5281 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5e9f709fce547f8a0c9b8843e827fb8: Generated new TSK 0
I20260806 18:33:58.723731 5281 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:33:58.731168 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 5164
2026-08-06T18:33:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1488 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:33:58 dist-test-slave-5bn2 krb5kdc[5296](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:33:58 dist-test-slave-5bn2 krb5kdc[5296](info): set up 2 sockets
Aug 06 18:33:58 dist-test-slave-5bn2 krb5kdc[5296](info): commencing operation
krb5kdc: starting...
W20260806 18:33:59.862975 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:33:59 dist-test-slave-5bn2 krb5kdc[5296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:33:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:33:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:33:59.989696 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45413
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:00.102129 5315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:00.102411 5315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:00.102470 5315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:00.106441 5315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:00.106508 5315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:00.106535 5315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:00.106555 5315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:00.106575 5315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:00.111927 5315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:00.113039 5315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:00.114074 5315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:00.122331 5322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:00.122354 5326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:00.122467 5315 server_base.cc:1034] running on GCE node
W20260806 18:34:00.122355 5324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:00.123198 5315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:00.124261 5315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:00.125439 5315 hybrid_clock.cc:648] HybridClock initialized: now 1786041240125421 us; error 29 us; skew 500 ppm
Aug 06 18:34:00 dist-test-slave-5bn2 krb5kdc[5296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041240, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:00.131532 5315 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:34:00.132515 5315 webserver.cc:460] Webserver started at http://127.0.0.1:40269/ using document root <none> and password file <none>
I20260806 18:34:00.133077 5315 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:00.133142 5315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:00.133365 5315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:00.135532 5315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "f440caf5bbdf49bb9a25d3132dbf7fe6"
format_stamp: "Formatted at 2026-08-06 18:34:00 on dist-test-slave-5bn2"
server_key: "e25fa681a41823169bc155b501ad4726"
server_key_iv: "fc833d76ba615f993265416a92f319b4"
server_key_version: "encryptionkey@0"
I20260806 18:34:00.136088 5315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "f440caf5bbdf49bb9a25d3132dbf7fe6"
format_stamp: "Formatted at 2026-08-06 18:34:00 on dist-test-slave-5bn2"
server_key: "e25fa681a41823169bc155b501ad4726"
server_key_iv: "fc833d76ba615f993265416a92f319b4"
server_key_version: "encryptionkey@0"
I20260806 18:34:00.139868 5315 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:34:00.142287 5341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:00.143510 5315 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:00.143625 5315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "f440caf5bbdf49bb9a25d3132dbf7fe6"
format_stamp: "Formatted at 2026-08-06 18:34:00 on dist-test-slave-5bn2"
server_key: "e25fa681a41823169bc155b501ad4726"
server_key_iv: "fc833d76ba615f993265416a92f319b4"
server_key_version: "encryptionkey@0"
I20260806 18:34:00.143731 5315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:00.148392 5315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:00.151526 5315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:00.151706 5315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:00.158777 5315 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45413
I20260806 18:34:00.158807 5414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45413 every 8 connection(s)
I20260806 18:34:00.159763 5315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:00.162524 5415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:00.166235 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 5315
I20260806 18:34:00.166370 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:34:00.166661 7439 external_mini_cluster.cc:1428] Setting key c8758cab8e32093cb1eb7f9f2b876d0c
I20260806 18:34:00.168483 5415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6: Bootstrap starting.
I20260806 18:34:00.171011 5415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:00.171785 5415 log.cc:824] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:00.172009 5420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:00.168110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42745 (local address 127.0.0.1:45413)
0806 18:34:00.168550 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:00.168569 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:00.168614 (+ 45us) server_negotiation.cc:407] Connection header received
0806 18:34:00.169263 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:00.169280 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:00.169565 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:00.169844 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:00.170724 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:00.170729 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:00.170741 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:00.170752 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:00.170808 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:00.170827 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:00.170959 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:00.171074 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 18:34:00.171241 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":119,"threads_started":1}
I20260806 18:34:00.174600 5415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6: No bootstrap required, opened a new log
I20260806 18:34:00.174824 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:00.178419 5420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:00.176824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42265 (local address 127.0.0.1:45413)
0806 18:34:00.176936 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:00.176939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:00.177175 (+ 236us) server_negotiation.cc:407] Connection header received
0806 18:34:00.177290 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:00.177292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:00.177323 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:00.177386 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:00.178074 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:00.178077 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:00.178079 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:00.178088 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:00.178112 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:00.178122 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:00.178186 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:00.178270 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 18:34:00.178302 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:34:00.178960 5415 raft_consensus.cc:348] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f440caf5bbdf49bb9a25d3132dbf7fe6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45413 } }
I20260806 18:34:00.179205 5415 raft_consensus.cc:374] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:00.179267 5415 raft_consensus.cc:729] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f440caf5bbdf49bb9a25d3132dbf7fe6, State: Initialized, Role: FOLLOWER
I20260806 18:34:00.179754 5415 consensus_queue.cc:260] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f440caf5bbdf49bb9a25d3132dbf7fe6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45413 } }
I20260806 18:34:00.179883 5415 raft_consensus.cc:388] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:00.179929 5415 raft_consensus.cc:482] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:00.180029 5415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:00.180964 5415 raft_consensus.cc:504] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f440caf5bbdf49bb9a25d3132dbf7fe6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45413 } }
I20260806 18:34:00.181284 5415 leader_election.cc:302] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f440caf5bbdf49bb9a25d3132dbf7fe6; no voters:
I20260806 18:34:00.181610 5415 leader_election.cc:288] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:00.181699 5430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:00.181924 5430 raft_consensus.cc:686] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6 [term 1 LEADER]: Becoming Leader. State: Replica: f440caf5bbdf49bb9a25d3132dbf7fe6, State: Running, Role: LEADER
I20260806 18:34:00.182260 5430 consensus_queue.cc:237] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f440caf5bbdf49bb9a25d3132dbf7fe6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45413 } }
I20260806 18:34:00.182811 5415 sys_catalog.cc:564] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:00.183995 5431 sys_catalog.cc:455] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f440caf5bbdf49bb9a25d3132dbf7fe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f440caf5bbdf49bb9a25d3132dbf7fe6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45413 } } }
I20260806 18:34:00.184136 5431 sys_catalog.cc:458] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:00.184000 5432 sys_catalog.cc:455] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f440caf5bbdf49bb9a25d3132dbf7fe6. Latest consensus state: current_term: 1 leader_uuid: "f440caf5bbdf49bb9a25d3132dbf7fe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f440caf5bbdf49bb9a25d3132dbf7fe6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45413 } } }
I20260806 18:34:00.184409 5432 sys_catalog.cc:458] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:00.184508 5442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:00.187474 5442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:00.192797 5442 catalog_manager.cc:1347] Generated new cluster ID: 38330c35a3394a079f782fe95b297493
I20260806 18:34:00.192873 5442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:00.218899 5442 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:00.219316 5442 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:00.226016 5442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f440caf5bbdf49bb9a25d3132dbf7fe6: Generated new TSK 0
I20260806 18:34:00.226999 5442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:00.239013 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 5315
2026-08-06T18:34:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:00 dist-test-slave-5bn2 krb5kdc[5464](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:00 dist-test-slave-5bn2 krb5kdc[5464](info): set up 2 sockets
Aug 06 18:34:00 dist-test-slave-5bn2 krb5kdc[5464](info): commencing operation
krb5kdc: starting...
W20260806 18:34:01.368798 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:34:01 dist-test-slave-5bn2 krb5kdc[5464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:34:01.496254 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46283
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:01.610033 5487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:01.610329 5487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:01.610371 5487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:01.614362 5487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:01.614441 5487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:01.614470 5487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:01.614491 5487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:01.614558 5487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:01.619978 5487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:01.621068 5487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:01.622352 5487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:01.631289 5494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:01.631284 5499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:01.631381 5487 server_base.cc:1034] running on GCE node
W20260806 18:34:01.631289 5495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:01.632128 5487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:01.633203 5487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:01.634388 5487 hybrid_clock.cc:648] HybridClock initialized: now 1786041241634364 us; error 42 us; skew 500 ppm
Aug 06 18:34:01 dist-test-slave-5bn2 krb5kdc[5464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041241, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:01.637092 5487 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:34:01.638041 5487 webserver.cc:460] Webserver started at http://127.0.0.1:33637/ using document root <none> and password file <none>
I20260806 18:34:01.638602 5487 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:01.638672 5487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:01.638911 5487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:01.640986 5487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "78f1e828b37446279c628639f539710f"
format_stamp: "Formatted at 2026-08-06 18:34:01 on dist-test-slave-5bn2"
server_key: "6ad4d16d4ce3c86d8101da2abe2ba423"
server_key_iv: "688197d4b6aaed8bbdb47a47e09c6ff5"
server_key_version: "encryptionkey@0"
I20260806 18:34:01.641542 5487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "78f1e828b37446279c628639f539710f"
format_stamp: "Formatted at 2026-08-06 18:34:01 on dist-test-slave-5bn2"
server_key: "6ad4d16d4ce3c86d8101da2abe2ba423"
server_key_iv: "688197d4b6aaed8bbdb47a47e09c6ff5"
server_key_version: "encryptionkey@0"
I20260806 18:34:01.645251 5487 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260806 18:34:01.647758 5509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:01.648983 5487 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260806 18:34:01.649116 5487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "78f1e828b37446279c628639f539710f"
format_stamp: "Formatted at 2026-08-06 18:34:01 on dist-test-slave-5bn2"
server_key: "6ad4d16d4ce3c86d8101da2abe2ba423"
server_key_iv: "688197d4b6aaed8bbdb47a47e09c6ff5"
server_key_version: "encryptionkey@0"
I20260806 18:34:01.649223 5487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:01.668563 5487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:01.671623 5487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:01.671814 5487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:01.678828 5487 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46283
I20260806 18:34:01.678874 5581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46283 every 8 connection(s)
I20260806 18:34:01.679821 5487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:01.682658 5582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:01.682894 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 5487
I20260806 18:34:01.683036 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:34:01.683388 7439 external_mini_cluster.cc:1428] Setting key 40fefb4766c9e247ab2bf00094018e09
I20260806 18:34:01.689765 5582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f: Bootstrap starting.
I20260806 18:34:01.692551 5582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f: Neither blocks nor log segments found. Creating new log.
Aug 06 18:34:01 dist-test-slave-5bn2 krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:34:01.693459 5582 log.cc:824] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:01.695643 5582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f: No bootstrap required, opened a new log
I20260806 18:34:01.697970 5585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:01.684863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:53001 (local address 127.0.0.1:46283)
0806 18:34:01.685309 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:01.685324 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:01.685362 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:34:01.686055 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:01.686073 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:01.686415 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:01.686717 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:01.687647 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:01.688643 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:01.689797 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:01.690327 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:01.690440 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:01.693269 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:01.693274 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:01.693288 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:01.693317 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:01.695044 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:01.695609 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:01.695614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:01.695618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:01.695749 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:01.696100 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:01.696103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:01.696104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:01.696493 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:01.696690 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:01.697026 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 18:34:01.697243 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":137,"threads_started":1}
I20260806 18:34:01.699517 5582 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78f1e828b37446279c628639f539710f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46283 } }
I20260806 18:34:01.699802 5582 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:01.699867 5582 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78f1e828b37446279c628639f539710f, State: Initialized, Role: FOLLOWER
I20260806 18:34:01.700345 5582 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78f1e828b37446279c628639f539710f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46283 } }
I20260806 18:34:01.700505 5582 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:01.700582 5582 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:01.700686 5582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:01.701115 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:01.701839 5582 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78f1e828b37446279c628639f539710f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46283 } }
I20260806 18:34:01.702231 5582 leader_election.cc:302] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78f1e828b37446279c628639f539710f; no voters:
I20260806 18:34:01.702581 5582 leader_election.cc:288] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:01.702720 5590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:01.702962 5590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f [term 1 LEADER]: Becoming Leader. State: Replica: 78f1e828b37446279c628639f539710f, State: Running, Role: LEADER
I20260806 18:34:01.703334 5590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78f1e828b37446279c628639f539710f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46283 } }
I20260806 18:34:01.704083 5582 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:01.705006 5596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78f1e828b37446279c628639f539710f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78f1e828b37446279c628639f539710f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46283 } } }
I20260806 18:34:01.705192 5596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:01.705472 5596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78f1e828b37446279c628639f539710f. Latest consensus state: current_term: 1 leader_uuid: "78f1e828b37446279c628639f539710f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78f1e828b37446279c628639f539710f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46283 } } }
I20260806 18:34:01.705613 5596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:01.706732 5607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:01.709791 5607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:01.712944 5585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:01.703443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:43101 (local address 127.0.0.1:46283)
0806 18:34:01.703568 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:01.703572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:01.703819 (+ 247us) server_negotiation.cc:407] Connection header received
0806 18:34:01.703932 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:01.703935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:01.703983 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:01.704063 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:01.704886 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:01.705673 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:01.706930 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:01.707270 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:01.707368 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:01.709012 (+ 1644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:01.709016 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:01.709018 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:01.709045 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:01.710748 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:01.711383 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:01.711388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:01.711390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:01.711482 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:01.711904 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:01.711908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:01.711910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:01.712153 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:01.712328 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:01.712660 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 18:34:01.712771 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:34:01.715617 5607 catalog_manager.cc:1347] Generated new cluster ID: 1d43c7f67e6343558a712a02e3bafe06
I20260806 18:34:01.715687 5607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:01.723664 5607 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:01.724884 5607 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:01.734431 5607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78f1e828b37446279c628639f539710f: Generated new TSK 0
I20260806 18:34:01.735314 5607 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:01.748343 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 5487
W20260806 18:34:01.758572 5461 connection.cc:535] client connection to 127.0.0.1:46283 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46283): Connection reset by peer (error 104)
2026-08-06T18:34:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1500 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:01 dist-test-slave-5bn2 krb5kdc[5623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:01 dist-test-slave-5bn2 krb5kdc[5623](info): set up 2 sockets
Aug 06 18:34:01 dist-test-slave-5bn2 krb5kdc[5623](info): commencing operation
krb5kdc: starting...
W20260806 18:34:02.864498 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:34:02 dist-test-slave-5bn2 krb5kdc[5623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:34:02.990376 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43571
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:03.105016 5644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:03.105314 5644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:03.105373 5644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:03.109475 5644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:03.109555 5644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:03.109580 5644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:03.109601 5644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:03.109623 5644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:03.115092 5644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:03.116245 5644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:03.117324 5644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:03.126081 5652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:03.126150 5644 server_base.cc:1034] running on GCE node
W20260806 18:34:03.126063 5655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:03.126063 5651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:03.126955 5644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:03.128187 5644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:03.129346 5644 hybrid_clock.cc:648] HybridClock initialized: now 1786041243129327 us; error 32 us; skew 500 ppm
Aug 06 18:34:03 dist-test-slave-5bn2 krb5kdc[5623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041243, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:03.132175 5644 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:34:03.133112 5644 webserver.cc:460] Webserver started at http://127.0.0.1:37951/ using document root <none> and password file <none>
I20260806 18:34:03.133663 5644 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:03.133726 5644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:03.133935 5644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:03.136051 5644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "f41df72d20b445d9be98187538428a12"
format_stamp: "Formatted at 2026-08-06 18:34:03 on dist-test-slave-5bn2"
server_key: "f1b76f89964e4e8e40a48205195b386c"
server_key_iv: "fd7173701f14ef04ee2cce8ceb36358c"
server_key_version: "encryptionkey@0"
I20260806 18:34:03.136598 5644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "f41df72d20b445d9be98187538428a12"
format_stamp: "Formatted at 2026-08-06 18:34:03 on dist-test-slave-5bn2"
server_key: "f1b76f89964e4e8e40a48205195b386c"
server_key_iv: "fd7173701f14ef04ee2cce8ceb36358c"
server_key_version: "encryptionkey@0"
I20260806 18:34:03.140290 5644 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:34:03.142730 5668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:03.144024 5644 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:03.144145 5644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "f41df72d20b445d9be98187538428a12"
format_stamp: "Formatted at 2026-08-06 18:34:03 on dist-test-slave-5bn2"
server_key: "f1b76f89964e4e8e40a48205195b386c"
server_key_iv: "fd7173701f14ef04ee2cce8ceb36358c"
server_key_version: "encryptionkey@0"
I20260806 18:34:03.144249 5644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:03.153129 5644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:03.156273 5644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:03.156466 5644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:03.163297 5644 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43571
I20260806 18:34:03.163312 5747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43571 every 8 connection(s)
I20260806 18:34:03.164242 5644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:03.167007 5748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:03.167248 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 5644
I20260806 18:34:03.167397 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:34:03.167733 7439 external_mini_cluster.cc:1428] Setting key db9d45a3bc6464a46a8ea82f33711246
I20260806 18:34:03.173760 5748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12: Bootstrap starting.
I20260806 18:34:03.176101 5748 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:03.176664 5751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:03.169054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:59513 (local address 127.0.0.1:43571)
0806 18:34:03.169460 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:03.169477 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:03.169512 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:34:03.170219 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:03.170236 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:03.170573 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:03.170840 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:03.171229 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:03.172255 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:03.173413 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:03.173878 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:03.174043 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:03.174972 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:03.174977 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:03.174990 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:03.175356 (+ 366us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:03.175410 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:03.175436 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:03.175556 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:03.175737 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 18:34:03.175871 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":270,"thread_start_us":114,"threads_started":1}
I20260806 18:34:03.176867 5748 log.cc:824] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:03.178808 5748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12: No bootstrap required, opened a new log
I20260806 18:34:03.179562 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:03.182608 5748 raft_consensus.cc:348] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41df72d20b445d9be98187538428a12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43571 } }
I20260806 18:34:03.182829 5748 raft_consensus.cc:374] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:03.182896 5748 raft_consensus.cc:729] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41df72d20b445d9be98187538428a12, State: Initialized, Role: FOLLOWER
I20260806 18:34:03.183377 5748 consensus_queue.cc:260] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41df72d20b445d9be98187538428a12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43571 } }
I20260806 18:34:03.183540 5748 raft_consensus.cc:388] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:03.183600 5748 raft_consensus.cc:482] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:03.183691 5748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:03.184736 5748 raft_consensus.cc:504] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41df72d20b445d9be98187538428a12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43571 } }
I20260806 18:34:03.185104 5748 leader_election.cc:302] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f41df72d20b445d9be98187538428a12; no voters:
I20260806 18:34:03.185460 5748 leader_election.cc:288] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:03.185532 5761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:03.185468 5751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:03.181493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:51853 (local address 127.0.0.1:43571)
0806 18:34:03.181599 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:03.181603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:03.181789 (+ 186us) server_negotiation.cc:407] Connection header received
0806 18:34:03.181898 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:03.181901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:03.181944 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:03.182008 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:03.182268 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:03.182789 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:03.183877 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:03.184122 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:03.184199 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:03.185007 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:03.185010 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:03.185012 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:03.185023 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:03.185047 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:03.185057 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:03.185126 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:03.185303 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 18:34:03.185338 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 18:34:03.185752 5761 raft_consensus.cc:686] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12 [term 1 LEADER]: Becoming Leader. State: Replica: f41df72d20b445d9be98187538428a12, State: Running, Role: LEADER
I20260806 18:34:03.186043 5761 consensus_queue.cc:237] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41df72d20b445d9be98187538428a12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43571 } }
I20260806 18:34:03.186802 5748 sys_catalog.cc:564] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:03.187213 5763 sys_catalog.cc:455] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f41df72d20b445d9be98187538428a12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41df72d20b445d9be98187538428a12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43571 } } }
I20260806 18:34:03.187546 5763 sys_catalog.cc:458] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:03.187440 5762 sys_catalog.cc:455] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f41df72d20b445d9be98187538428a12. Latest consensus state: current_term: 1 leader_uuid: "f41df72d20b445d9be98187538428a12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41df72d20b445d9be98187538428a12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43571 } } }
I20260806 18:34:03.187876 5762 sys_catalog.cc:458] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:03.188175 5770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:03.191614 5770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:03.197000 5770 catalog_manager.cc:1347] Generated new cluster ID: ee018d0987864dde8bdddb451f2fe273
I20260806 18:34:03.197074 5770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:03.227221 5770 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:03.228317 5770 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:03.239697 5770 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f41df72d20b445d9be98187538428a12: Generated new TSK 0
I20260806 18:34:03.240469 5770 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:03.253003 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 5644
W20260806 18:34:03.261847 5620 connection.cc:535] client connection to 127.0.0.1:43571 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43571): Connection reset by peer (error 104)
2026-08-06T18:34:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1502 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:03 dist-test-slave-5bn2 krb5kdc[5787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:03 dist-test-slave-5bn2 krb5kdc[5787](info): set up 2 sockets
Aug 06 18:34:03 dist-test-slave-5bn2 krb5kdc[5787](info): commencing operation
krb5kdc: starting...
W20260806 18:34:04.389535 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:34:04 dist-test-slave-5bn2 krb5kdc[5787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:34:04.517922 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36019
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:04.636233 5802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:04.636502 5802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:04.636550 5802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:04.640569 5802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:04.640632 5802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:04.640657 5802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:04.640688 5802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:04.640714 5802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:04.646497 5802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:04.647666 5802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:04.648699 5802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:04.657569 5807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:04.657802 5810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:04.657843 5802 server_base.cc:1034] running on GCE node
W20260806 18:34:04.657963 5808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:04.658563 5802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:04.659827 5802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:04.660998 5802 hybrid_clock.cc:648] HybridClock initialized: now 1786041244660972 us; error 40 us; skew 500 ppm
Aug 06 18:34:04 dist-test-slave-5bn2 krb5kdc[5787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041244, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:04.664078 5802 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:34:04.665071 5802 webserver.cc:460] Webserver started at http://127.0.0.1:32817/ using document root <none> and password file <none>
I20260806 18:34:04.665649 5802 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:04.665714 5802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:04.665915 5802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:04.668084 5802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "b07cfbf4de384408910bbfd72e2d6191"
format_stamp: "Formatted at 2026-08-06 18:34:04 on dist-test-slave-5bn2"
server_key: "c25577fbb3a36de8a97eb79436e2c237"
server_key_iv: "c916158bd4e0d9769a6852fc1f583464"
server_key_version: "encryptionkey@0"
I20260806 18:34:04.668665 5802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "b07cfbf4de384408910bbfd72e2d6191"
format_stamp: "Formatted at 2026-08-06 18:34:04 on dist-test-slave-5bn2"
server_key: "c25577fbb3a36de8a97eb79436e2c237"
server_key_iv: "c916158bd4e0d9769a6852fc1f583464"
server_key_version: "encryptionkey@0"
I20260806 18:34:04.672537 5802 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:34:04.675112 5827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:04.676435 5802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:04.676565 5802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "b07cfbf4de384408910bbfd72e2d6191"
format_stamp: "Formatted at 2026-08-06 18:34:04 on dist-test-slave-5bn2"
server_key: "c25577fbb3a36de8a97eb79436e2c237"
server_key_iv: "c916158bd4e0d9769a6852fc1f583464"
server_key_version: "encryptionkey@0"
I20260806 18:34:04.676673 5802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:04.681226 5802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:04.684051 5802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:04.684218 5802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:04.691066 5802 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36019
I20260806 18:34:04.691078 5908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36019 every 8 connection(s)
I20260806 18:34:04.692005 5802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:04.694751 5909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:04.695094 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 5802
I20260806 18:34:04.695245 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:34:04.695597 7439 external_mini_cluster.cc:1428] Setting key e87f5dd1998947c283549dbe1cc8e81d
I20260806 18:34:04.700663 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:04.696909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:55571 (local address 127.0.0.1:36019)
0806 18:34:04.697316 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:04.697332 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:04.697368 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:34:04.698075 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:04.698092 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:04.698368 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:04.698649 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:04.699491 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:04.699496 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:04.699507 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:04.699518 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:04.699578 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:04.699597 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:04.699703 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:04.699808 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 18:34:04.699955 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":113,"threads_started":1}
I20260806 18:34:04.701545 5909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191: Bootstrap starting.
I20260806 18:34:04.703284 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:04.704033 5909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:04.704932 5909 log.cc:824] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:04.706887 5909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191: No bootstrap required, opened a new log
I20260806 18:34:04.707057 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:04.705560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:56685 (local address 127.0.0.1:36019)
0806 18:34:04.705683 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:04.705687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:04.705840 (+ 153us) server_negotiation.cc:407] Connection header received
0806 18:34:04.705940 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:04.705943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:04.705975 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:04.706035 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:04.706692 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:04.706696 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:04.706698 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:04.706710 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:04.706736 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:04.706746 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:04.706815 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:04.706905 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 18:34:04.706946 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:34:04.710588 5909 raft_consensus.cc:348] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b07cfbf4de384408910bbfd72e2d6191" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36019 } }
I20260806 18:34:04.710878 5909 raft_consensus.cc:374] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:04.710944 5909 raft_consensus.cc:729] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b07cfbf4de384408910bbfd72e2d6191, State: Initialized, Role: FOLLOWER
I20260806 18:34:04.711352 5909 consensus_queue.cc:260] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b07cfbf4de384408910bbfd72e2d6191" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36019 } }
I20260806 18:34:04.711475 5909 raft_consensus.cc:388] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:04.711536 5909 raft_consensus.cc:482] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:04.711627 5909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:04.712491 5909 raft_consensus.cc:504] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b07cfbf4de384408910bbfd72e2d6191" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36019 } }
I20260806 18:34:04.712813 5909 leader_election.cc:302] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b07cfbf4de384408910bbfd72e2d6191; no voters:
I20260806 18:34:04.713115 5909 leader_election.cc:288] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:04.713213 5924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:04.713411 5924 raft_consensus.cc:686] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191 [term 1 LEADER]: Becoming Leader. State: Replica: b07cfbf4de384408910bbfd72e2d6191, State: Running, Role: LEADER
I20260806 18:34:04.713778 5924 consensus_queue.cc:237] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b07cfbf4de384408910bbfd72e2d6191" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36019 } }
I20260806 18:34:04.714301 5909 sys_catalog.cc:564] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:04.715538 5926 sys_catalog.cc:455] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b07cfbf4de384408910bbfd72e2d6191. Latest consensus state: current_term: 1 leader_uuid: "b07cfbf4de384408910bbfd72e2d6191" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b07cfbf4de384408910bbfd72e2d6191" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36019 } } }
I20260806 18:34:04.715548 5925 sys_catalog.cc:455] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b07cfbf4de384408910bbfd72e2d6191" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b07cfbf4de384408910bbfd72e2d6191" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36019 } } }
I20260806 18:34:04.715673 5926 sys_catalog.cc:458] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:04.715677 5925 sys_catalog.cc:458] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:04.716079 5939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:04.718781 5939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:04.723994 5939 catalog_manager.cc:1347] Generated new cluster ID: f8dc5c76a1db401eac24b43e3c0eda8b
I20260806 18:34:04.724071 5939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:04.742511 5939 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:04.742938 5939 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:04.746877 5939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b07cfbf4de384408910bbfd72e2d6191: Generated new TSK 0
I20260806 18:34:04.747560 5939 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:04.752748 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 5802
2026-08-06T18:34:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1502 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:04 dist-test-slave-5bn2 krb5kdc[5953](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:04 dist-test-slave-5bn2 krb5kdc[5953](info): set up 2 sockets
Aug 06 18:34:04 dist-test-slave-5bn2 krb5kdc[5953](info): commencing operation
krb5kdc: starting...
W20260806 18:34:05.875556 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:34:05 dist-test-slave-5bn2 krb5kdc[5953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:34:06.005486 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44807
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:06.118392 5970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:06.118656 5970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:06.118696 5970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:06.122658 5970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:06.122726 5970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:06.122745 5970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:06.122759 5970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:06.122773 5970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:06.128110 5970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44807
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:06.129124 5970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:06.130198 5970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:34:06.138696 5970 server_base.cc:1034] running on GCE node
W20260806 18:34:06.138640 5983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:06.138622 5979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:06.138660 5978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:06.139508 5970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:06.140573 5970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:06.141750 5970 hybrid_clock.cc:648] HybridClock initialized: now 1786041246141720 us; error 41 us; skew 500 ppm
Aug 06 18:34:06 dist-test-slave-5bn2 krb5kdc[5953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041246, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:06.147396 5970 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:34:06.148342 5970 webserver.cc:460] Webserver started at http://127.0.0.1:33125/ using document root <none> and password file <none>
I20260806 18:34:06.148885 5970 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:06.148964 5970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:06.149178 5970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:06.151288 5970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "adddf788cffe4cccaa3153ee44c67d84"
format_stamp: "Formatted at 2026-08-06 18:34:06 on dist-test-slave-5bn2"
server_key: "3a6e6d122766da2ab358076e1cad8ec8"
server_key_iv: "6e8c1aaec17a77c0cd5cd12d78bbf616"
server_key_version: "encryptionkey@0"
I20260806 18:34:06.151830 5970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "adddf788cffe4cccaa3153ee44c67d84"
format_stamp: "Formatted at 2026-08-06 18:34:06 on dist-test-slave-5bn2"
server_key: "3a6e6d122766da2ab358076e1cad8ec8"
server_key_iv: "6e8c1aaec17a77c0cd5cd12d78bbf616"
server_key_version: "encryptionkey@0"
I20260806 18:34:06.155498 5970 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:34:06.157837 5992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:06.159052 5970 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260806 18:34:06.159168 5970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "adddf788cffe4cccaa3153ee44c67d84"
format_stamp: "Formatted at 2026-08-06 18:34:06 on dist-test-slave-5bn2"
server_key: "3a6e6d122766da2ab358076e1cad8ec8"
server_key_iv: "6e8c1aaec17a77c0cd5cd12d78bbf616"
server_key_version: "encryptionkey@0"
I20260806 18:34:06.159267 5970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:06.181998 5970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:06.185228 5970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:06.185417 5970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:06.192668 5970 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44807
I20260806 18:34:06.192710 6067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44807 every 8 connection(s)
I20260806 18:34:06.193629 5970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:06.196427 6069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:06.201984 6069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84: Bootstrap starting.
I20260806 18:34:06.202545 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 5970
I20260806 18:34:06.202704 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:34:06.203034 7439 external_mini_cluster.cc:1428] Setting key 104447380d4cf00099722d443687a4e2
I20260806 18:34:06.204957 6069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:06.205664 6069 log.cc:824] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:06.207849 6069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84: No bootstrap required, opened a new log
I20260806 18:34:06.212060 6069 raft_consensus.cc:348] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddf788cffe4cccaa3153ee44c67d84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44807 } }
I20260806 18:34:06.212276 6069 raft_consensus.cc:374] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Aug 06 18:34:06 dist-test-slave-5bn2 krb5kdc[5953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:34:06.212309 6069 raft_consensus.cc:729] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adddf788cffe4cccaa3153ee44c67d84, State: Initialized, Role: FOLLOWER
I20260806 18:34:06.212738 6069 consensus_queue.cc:260] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddf788cffe4cccaa3153ee44c67d84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44807 } }
I20260806 18:34:06.212872 6069 raft_consensus.cc:388] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:06.212939 6069 raft_consensus.cc:482] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:06.213037 6069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:06.213874 6069 raft_consensus.cc:504] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddf788cffe4cccaa3153ee44c67d84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44807 } }
I20260806 18:34:06.214184 6069 leader_election.cc:302] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adddf788cffe4cccaa3153ee44c67d84; no voters:
I20260806 18:34:06.214485 6069 leader_election.cc:288] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:06.214639 6076 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:06.214957 6076 raft_consensus.cc:686] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84 [term 1 LEADER]: Becoming Leader. State: Replica: adddf788cffe4cccaa3153ee44c67d84, State: Running, Role: LEADER
I20260806 18:34:06.215348 6076 consensus_queue.cc:237] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddf788cffe4cccaa3153ee44c67d84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44807 } }
I20260806 18:34:06.215770 6069 sys_catalog.cc:564] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:06.218111 6079 sys_catalog.cc:455] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adddf788cffe4cccaa3153ee44c67d84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddf788cffe4cccaa3153ee44c67d84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44807 } } }
I20260806 18:34:06.218225 6079 sys_catalog.cc:458] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:06.218624 6074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:06.204438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:54225 (local address 127.0.0.1:44807)
0806 18:34:06.204879 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:06.204902 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:06.204938 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:34:06.205599 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:06.205617 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:06.206005 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:06.206378 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:06.207242 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:06.208276 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:06.209456 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:06.209942 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:06.210058 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:06.212902 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:06.212909 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:06.212926 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:06.212970 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:06.214954 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:06.215602 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:06.215609 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:06.215614 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:06.215755 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:06.216151 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:06.216154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:06.216156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:06.216673 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:06.216892 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:06.217346 (+ 454us) server_negotiation.cc:299] Negotiation successful
0806 18:34:06.217596 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":120,"threads_started":1}
I20260806 18:34:06.218590 6080 sys_catalog.cc:455] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adddf788cffe4cccaa3153ee44c67d84. Latest consensus state: current_term: 1 leader_uuid: "adddf788cffe4cccaa3153ee44c67d84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddf788cffe4cccaa3153ee44c67d84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44807 } } }
I20260806 18:34:06.219769 6080 sys_catalog.cc:458] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84 [sys.catalog]: This master's current role is: LEADER
W20260806 18:34:06.221004 6098 catalog_manager.cc:1558] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:34:06.221115 6098 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:34:06.221201 6100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:06.221556 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:06.222056 6100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:06.227824 6100 catalog_manager.cc:1347] Generated new cluster ID: c953505b3cca4d41a519403a1bbf826d
I20260806 18:34:06.227893 6100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:06.233094 6074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:06.223542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:54653 (local address 127.0.0.1:44807)
0806 18:34:06.223644 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:06.223648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:06.224215 (+ 567us) server_negotiation.cc:407] Connection header received
0806 18:34:06.224340 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:06.224343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:06.224393 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:06.224465 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:06.225310 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:06.226304 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:06.227508 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:06.227864 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:06.227953 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:06.229739 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:06.229743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:06.229746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:06.229772 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:06.231378 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:06.231874 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:06.231877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:06.231879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:06.231952 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:06.232269 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:06.232272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:06.232274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:06.232454 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:06.232598 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:06.232842 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 18:34:06.232933 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:34:06.252502 6100 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:06.253451 6100 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:06.265838 6100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adddf788cffe4cccaa3153ee44c67d84: Generated new TSK 0
I20260806 18:34:06.266527 6100 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:06.269985 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 5970
W20260806 18:34:06.282362 5950 connection.cc:535] client connection to 127.0.0.1:44807 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44807): Connection reset by peer (error 104)
2026-08-06T18:34:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1518 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:06 dist-test-slave-5bn2 krb5kdc[6114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:06 dist-test-slave-5bn2 krb5kdc[6114](info): set up 2 sockets
Aug 06 18:34:06 dist-test-slave-5bn2 krb5kdc[6114](info): commencing operation
krb5kdc: starting...
W20260806 18:34:07.401103 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:34:07 dist-test-slave-5bn2 krb5kdc[6114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:34:07.530045 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45963
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:07.643829 6133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:07.644232 6133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:07.644301 6133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:07.648667 6133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:07.648737 6133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:07.648756 6133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:07.648772 6133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:07.648785 6133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:07.654403 6133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45963
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:07.655550 6133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:07.656563 6133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:07.665149 6141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:07.665155 6143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:07.665182 6139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:07.665493 6133 server_base.cc:1034] running on GCE node
I20260806 18:34:07.666074 6133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:07.667362 6133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:07.668529 6133 hybrid_clock.cc:648] HybridClock initialized: now 1786041247668515 us; error 28 us; skew 500 ppm
Aug 06 18:34:07 dist-test-slave-5bn2 krb5kdc[6114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041247, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:07.671453 6133 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:34:07.672394 6133 webserver.cc:460] Webserver started at http://127.0.0.1:38649/ using document root <none> and password file <none>
I20260806 18:34:07.672961 6133 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:07.673025 6133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:07.673220 6133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:07.675390 6133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "a590fc6d63b54d01820f8c4d47bd8e72"
format_stamp: "Formatted at 2026-08-06 18:34:07 on dist-test-slave-5bn2"
server_key: "8f722f7d22077187f9cfcf74251f426e"
server_key_iv: "57d9ee3f4ce2eda0988ec3e8adfecea0"
server_key_version: "encryptionkey@0"
I20260806 18:34:07.675974 6133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "a590fc6d63b54d01820f8c4d47bd8e72"
format_stamp: "Formatted at 2026-08-06 18:34:07 on dist-test-slave-5bn2"
server_key: "8f722f7d22077187f9cfcf74251f426e"
server_key_iv: "57d9ee3f4ce2eda0988ec3e8adfecea0"
server_key_version: "encryptionkey@0"
I20260806 18:34:07.679800 6133 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:34:07.682215 6154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:07.683486 6133 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:07.683606 6133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "a590fc6d63b54d01820f8c4d47bd8e72"
format_stamp: "Formatted at 2026-08-06 18:34:07 on dist-test-slave-5bn2"
server_key: "8f722f7d22077187f9cfcf74251f426e"
server_key_iv: "57d9ee3f4ce2eda0988ec3e8adfecea0"
server_key_version: "encryptionkey@0"
I20260806 18:34:07.683705 6133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:07.696003 6133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:07.699076 6133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:07.699261 6133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:07.706167 6133 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45963
I20260806 18:34:07.706230 6236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45963 every 8 connection(s)
I20260806 18:34:07.707190 6133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:07.710129 6238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:07.715937 6238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72: Bootstrap starting.
I20260806 18:34:07.717221 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 6133
I20260806 18:34:07.717345 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:34:07.717623 7439 external_mini_cluster.cc:1428] Setting key a5580557082d5badd3e5e55e0f356844
I20260806 18:34:07.718791 6238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:07.719769 6238 log.cc:824] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:07.722086 6238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72: No bootstrap required, opened a new log
I20260806 18:34:07.726482 6238 raft_consensus.cc:348] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a590fc6d63b54d01820f8c4d47bd8e72" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45963 } }
I20260806 18:34:07.726843 6238 raft_consensus.cc:374] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:07.726941 6238 raft_consensus.cc:729] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a590fc6d63b54d01820f8c4d47bd8e72, State: Initialized, Role: FOLLOWER
I20260806 18:34:07.727459 6238 consensus_queue.cc:260] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a590fc6d63b54d01820f8c4d47bd8e72" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45963 } }
I20260806 18:34:07.727633 6238 raft_consensus.cc:388] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:07.727708 6238 raft_consensus.cc:482] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:07.727809 6238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:07.727922 6244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:07.718770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:48533 (local address 127.0.0.1:45963)
0806 18:34:07.719324 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:07.719346 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:07.719395 (+ 49us) server_negotiation.cc:407] Connection header received
0806 18:34:07.720301 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:07.720324 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:07.720716 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:07.721110 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:07.721555 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:07.723077 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:07.723988 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:07.724742 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:07.724944 (+ 202us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:07.725772 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:07.725779 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:07.725797 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:07.726327 (+ 530us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:07.726403 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:07.726437 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:07.726598 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:07.726703 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 18:34:07.726885 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":159,"threads_started":1}
I20260806 18:34:07.728921 6238 raft_consensus.cc:504] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a590fc6d63b54d01820f8c4d47bd8e72" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45963 } }
I20260806 18:34:07.729298 6238 leader_election.cc:302] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a590fc6d63b54d01820f8c4d47bd8e72; no voters:
I20260806 18:34:07.729674 6238 leader_election.cc:288] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:07.729770 6246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:07.730057 6246 raft_consensus.cc:686] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72 [term 1 LEADER]: Becoming Leader. State: Replica: a590fc6d63b54d01820f8c4d47bd8e72, State: Running, Role: LEADER
I20260806 18:34:07.730466 6246 consensus_queue.cc:237] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a590fc6d63b54d01820f8c4d47bd8e72" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45963 } }
I20260806 18:34:07.731158 6238 sys_catalog.cc:564] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:07.731480 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:07.732324 6247 sys_catalog.cc:455] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a590fc6d63b54d01820f8c4d47bd8e72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a590fc6d63b54d01820f8c4d47bd8e72" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45963 } } }
I20260806 18:34:07.732332 6248 sys_catalog.cc:455] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a590fc6d63b54d01820f8c4d47bd8e72. Latest consensus state: current_term: 1 leader_uuid: "a590fc6d63b54d01820f8c4d47bd8e72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a590fc6d63b54d01820f8c4d47bd8e72" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45963 } } }
I20260806 18:34:07.732461 6247 sys_catalog.cc:458] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:07.732470 6248 sys_catalog.cc:458] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72 [sys.catalog]: This master's current role is: LEADER
W20260806 18:34:07.736785 6281 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:34:07.736920 6281 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:34:07.737048 6264 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:07.738059 6264 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:07.738628 6244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:07.734209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:36019 (local address 127.0.0.1:45963)
0806 18:34:07.734347 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:07.734351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:07.734456 (+ 105us) server_negotiation.cc:407] Connection header received
0806 18:34:07.734571 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:07.734574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:07.734621 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:07.734698 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:07.734999 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:07.735736 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:07.736905 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:07.737263 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:07.737358 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:07.738172 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:07.738176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:07.738178 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:07.738192 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:07.738219 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:07.738229 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:07.738301 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:07.738456 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 18:34:07.738494 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:34:07.743992 6264 catalog_manager.cc:1347] Generated new cluster ID: 09ce2da5ec4a481e985bf8d1e4734aef
I20260806 18:34:07.744056 6264 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:07.756276 6264 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:07.757265 6264 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:07.763263 6264 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a590fc6d63b54d01820f8c4d47bd8e72: Generated new TSK 0
I20260806 18:34:07.763917 6264 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:07.769389 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 6133
W20260806 18:34:07.786799 6110 connection.cc:535] client connection to 127.0.0.1:45963 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45963): Connection reset by peer (error 104)
2026-08-06T18:34:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1504 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:07 dist-test-slave-5bn2 krb5kdc[6292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:07 dist-test-slave-5bn2 krb5kdc[6292](info): set up 2 sockets
Aug 06 18:34:07 dist-test-slave-5bn2 krb5kdc[6292](info): commencing operation
krb5kdc: starting...
W20260806 18:34:08.888748 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:34:08 dist-test-slave-5bn2 krb5kdc[6292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:34:09.023416 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44787
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:09.149960 6309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:09.150249 6309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:09.150291 6309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:09.154246 6309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:09.154320 6309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:09.154337 6309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:09.154352 6309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:09.154366 6309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:09.159633 6309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44787
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:09.160658 6309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:09.161623 6309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:09.170249 6316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:09.170243 6315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:09.170245 6318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:09.170528 6309 server_base.cc:1034] running on GCE node
I20260806 18:34:09.171229 6309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:09.172318 6309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:09.173483 6309 hybrid_clock.cc:648] HybridClock initialized: now 1786041249173464 us; error 31 us; skew 500 ppm
Aug 06 18:34:09 dist-test-slave-5bn2 krb5kdc[6292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041249, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:09.176311 6309 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:34:09.177253 6309 webserver.cc:460] Webserver started at http://127.0.0.1:37919/ using document root <none> and password file <none>
I20260806 18:34:09.177771 6309 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:09.177829 6309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:09.178010 6309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:09.180119 6309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "4da956a586b74259b7421b885bfe5ef2"
format_stamp: "Formatted at 2026-08-06 18:34:09 on dist-test-slave-5bn2"
server_key: "efccf1fe3f11fcddfe4aa1600102bd6b"
server_key_iv: "07a3b396ac2b844317a6e43724faef86"
server_key_version: "encryptionkey@0"
I20260806 18:34:09.180691 6309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "4da956a586b74259b7421b885bfe5ef2"
format_stamp: "Formatted at 2026-08-06 18:34:09 on dist-test-slave-5bn2"
server_key: "efccf1fe3f11fcddfe4aa1600102bd6b"
server_key_iv: "07a3b396ac2b844317a6e43724faef86"
server_key_version: "encryptionkey@0"
I20260806 18:34:09.184345 6309 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:34:09.186677 6330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:09.187907 6309 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:09.188037 6309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "4da956a586b74259b7421b885bfe5ef2"
format_stamp: "Formatted at 2026-08-06 18:34:09 on dist-test-slave-5bn2"
server_key: "efccf1fe3f11fcddfe4aa1600102bd6b"
server_key_iv: "07a3b396ac2b844317a6e43724faef86"
server_key_version: "encryptionkey@0"
I20260806 18:34:09.188136 6309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:09.192798 6309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:09.195749 6309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:09.195925 6309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:09.202963 6309 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44787
I20260806 18:34:09.202993 6401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44787 every 8 connection(s)
I20260806 18:34:09.203886 6309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:09.206693 6402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:09.211218 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 6309
I20260806 18:34:09.211365 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:34:09.211627 7439 external_mini_cluster.cc:1428] Setting key c5e6dbd4153bd6f7d4608b4a2b289741
I20260806 18:34:09.212574 6402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2: Bootstrap starting.
I20260806 18:34:09.215409 6402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:09.216342 6402 log.cc:824] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:09.217283 6405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:09.212987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:42957 (local address 127.0.0.1:44787)
0806 18:34:09.213448 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:09.213467 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:09.213509 (+ 42us) server_negotiation.cc:407] Connection header received
0806 18:34:09.214376 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:09.214400 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:09.214762 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:09.215149 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:09.215893 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:09.215899 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:09.215914 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:09.215926 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:09.215990 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:09.216018 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:09.216125 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:09.216194 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 18:34:09.216401 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":107,"threads_started":1}
I20260806 18:34:09.218446 6402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2: No bootstrap required, opened a new log
I20260806 18:34:09.220253 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:09.223021 6402 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da956a586b74259b7421b885bfe5ef2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44787 } }
I20260806 18:34:09.223315 6402 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:09.223399 6402 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4da956a586b74259b7421b885bfe5ef2, State: Initialized, Role: FOLLOWER
I20260806 18:34:09.223887 6402 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da956a586b74259b7421b885bfe5ef2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44787 } }
I20260806 18:34:09.224066 6402 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:09.224140 6402 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:09.224189 6405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:09.222561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:55909 (local address 127.0.0.1:44787)
0806 18:34:09.222683 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:09.222687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:09.222889 (+ 202us) server_negotiation.cc:407] Connection header received
0806 18:34:09.223003 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:09.223005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:09.223042 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:09.223129 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:09.223829 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:09.223833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:09.223835 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:09.223848 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:09.223879 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:09.223891 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:09.223975 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:09.224034 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 18:34:09.224075 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:34:09.224242 6402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:09.225373 6402 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da956a586b74259b7421b885bfe5ef2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44787 } }
I20260806 18:34:09.225706 6402 leader_election.cc:302] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4da956a586b74259b7421b885bfe5ef2; no voters:
I20260806 18:34:09.226032 6402 leader_election.cc:288] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:09.226116 6417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:09.226328 6417 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2 [term 1 LEADER]: Becoming Leader. State: Replica: 4da956a586b74259b7421b885bfe5ef2, State: Running, Role: LEADER
I20260806 18:34:09.226653 6417 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da956a586b74259b7421b885bfe5ef2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44787 } }
I20260806 18:34:09.227314 6402 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:09.228384 6419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4da956a586b74259b7421b885bfe5ef2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da956a586b74259b7421b885bfe5ef2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44787 } } }
I20260806 18:34:09.228384 6423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4da956a586b74259b7421b885bfe5ef2. Latest consensus state: current_term: 1 leader_uuid: "4da956a586b74259b7421b885bfe5ef2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da956a586b74259b7421b885bfe5ef2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44787 } } }
I20260806 18:34:09.228535 6419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:09.228535 6423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:09.228921 6434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:09.232353 6434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:09.238170 6434 catalog_manager.cc:1347] Generated new cluster ID: 5df6c5e0ad36472e804e62ae77147c58
I20260806 18:34:09.238246 6434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:09.250342 6434 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:09.250669 6434 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:09.256285 6434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4da956a586b74259b7421b885bfe5ef2: Generated new TSK 0
I20260806 18:34:09.256955 6434 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:09.260592 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 6309
2026-08-06T18:34:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1484 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:09 dist-test-slave-5bn2 krb5kdc[6450](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:09 dist-test-slave-5bn2 krb5kdc[6450](info): set up 2 sockets
Aug 06 18:34:09 dist-test-slave-5bn2 krb5kdc[6450](info): commencing operation
krb5kdc: starting...
W20260806 18:34:10.379920 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:34:10 dist-test-slave-5bn2 krb5kdc[6450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:34:10.507962 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:36217
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:38611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:36217
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:10.623999 6466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:10.624284 6466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:10.624341 6466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:10.628342 6466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:10.628412 6466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:10.628439 6466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:10.628459 6466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:10.628484 6466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:10.633862 6466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:38611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:36217
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:36217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:10.635046 6466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:10.636140 6466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:10.645083 6479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:10.645083 6473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:10.645218 6466 server_base.cc:1034] running on GCE node
W20260806 18:34:10.645460 6477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:10.646030 6466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:10.647367 6466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:10.648531 6466 hybrid_clock.cc:648] HybridClock initialized: now 1786041250648516 us; error 33 us; skew 500 ppm
Aug 06 18:34:10 dist-test-slave-5bn2 krb5kdc[6450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041250, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:10.651382 6466 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:34:10.652345 6466 webserver.cc:460] Webserver started at http://127.7.67.254:43143/ using document root <none> and password file <none>
I20260806 18:34:10.652890 6466 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:10.652951 6466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:10.653127 6466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:10.655781 6466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "3e3dfd80c7154cd9a3d62bf7ac69c3f3"
format_stamp: "Formatted at 2026-08-06 18:34:10 on dist-test-slave-5bn2"
server_key: "a72536a4167a4a7c3d36786204eabf3f"
server_key_iv: "8e519305e08bbb8e20b2668845ee1d03"
server_key_version: "encryptionkey@0"
I20260806 18:34:10.656378 6466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "3e3dfd80c7154cd9a3d62bf7ac69c3f3"
format_stamp: "Formatted at 2026-08-06 18:34:10 on dist-test-slave-5bn2"
server_key: "a72536a4167a4a7c3d36786204eabf3f"
server_key_iv: "8e519305e08bbb8e20b2668845ee1d03"
server_key_version: "encryptionkey@0"
I20260806 18:34:10.660205 6466 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:34:10.662593 6489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:10.663789 6466 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:10.663924 6466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "3e3dfd80c7154cd9a3d62bf7ac69c3f3"
format_stamp: "Formatted at 2026-08-06 18:34:10 on dist-test-slave-5bn2"
server_key: "a72536a4167a4a7c3d36786204eabf3f"
server_key_iv: "8e519305e08bbb8e20b2668845ee1d03"
server_key_version: "encryptionkey@0"
I20260806 18:34:10.664018 6466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:10.677822 6466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:10.680974 6466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:10.681151 6466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:10.688253 6466 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:36217
I20260806 18:34:10.688265 6559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:36217 every 8 connection(s)
I20260806 18:34:10.689200 6466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:10.692018 6560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:10.695706 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 6466
I20260806 18:34:10.695858 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:34:10.696141 7439 external_mini_cluster.cc:1428] Setting key 8d0f1c8e3c506056171c52482ec09515
I20260806 18:34:10.697773 6560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3: Bootstrap starting.
I20260806 18:34:10.700300 6560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:10.701041 6560 log.cc:824] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:10.703264 6560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3: No bootstrap required, opened a new log
Aug 06 18:34:10 dist-test-slave-5bn2 krb5kdc[6450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:34:10.707628 6560 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3dfd80c7154cd9a3d62bf7ac69c3f3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 36217 } }
I20260806 18:34:10.707860 6560 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:10.707912 6560 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3dfd80c7154cd9a3d62bf7ac69c3f3, State: Initialized, Role: FOLLOWER
I20260806 18:34:10.708411 6560 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3dfd80c7154cd9a3d62bf7ac69c3f3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 36217 } }
I20260806 18:34:10.708549 6560 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:10.708609 6560 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:10.708710 6560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:10.709596 6560 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3dfd80c7154cd9a3d62bf7ac69c3f3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 36217 } }
I20260806 18:34:10.709944 6560 leader_election.cc:302] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e3dfd80c7154cd9a3d62bf7ac69c3f3; no voters:
I20260806 18:34:10.710317 6560 leader_election.cc:288] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:10.710421 6568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:10.710625 6568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3 [term 1 LEADER]: Becoming Leader. State: Replica: 3e3dfd80c7154cd9a3d62bf7ac69c3f3, State: Running, Role: LEADER
I20260806 18:34:10.711030 6568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3dfd80c7154cd9a3d62bf7ac69c3f3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 36217 } }
I20260806 18:34:10.711112 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.697598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50983 (local address 127.7.67.254:36217)
0806 18:34:10.698003 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:10.698019 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:10.698053 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:34:10.698695 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:10.698711 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:10.699091 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:10.699371 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:10.700355 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.701430 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.702628 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.703150 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.703267 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.706085 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:10.706091 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:10.706104 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:10.706132 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:10.707866 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:10.708400 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:10.708405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:10.708409 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:10.708528 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:10.708876 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:10.708879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:10.708881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:10.709283 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:10.709496 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:10.709838 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 18:34:10.710057 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":115,"threads_started":1}
I20260806 18:34:10.711684 6560 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:10.712677 6570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e3dfd80c7154cd9a3d62bf7ac69c3f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3dfd80c7154cd9a3d62bf7ac69c3f3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 36217 } } }
I20260806 18:34:10.712801 6570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:10.713101 6572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e3dfd80c7154cd9a3d62bf7ac69c3f3. Latest consensus state: current_term: 1 leader_uuid: "3e3dfd80c7154cd9a3d62bf7ac69c3f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3dfd80c7154cd9a3d62bf7ac69c3f3" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 36217 } } }
I20260806 18:34:10.713191 6576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:10.713267 6572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:10.714632 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:10.716600 6576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:10.722677 6576 catalog_manager.cc:1347] Generated new cluster ID: 05ee23fcda4a492fbc646fac494a7a6d
I20260806 18:34:10.722760 6576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:10.730227 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.718726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45079 (local address 127.7.67.254:36217)
0806 18:34:10.718869 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:10.718874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:10.723101 (+ 4227us) server_negotiation.cc:407] Connection header received
0806 18:34:10.723219 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:10.723222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:10.723268 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:10.723333 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:10.724007 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.724555 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.725361 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.725604 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.725693 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.727065 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:10.727069 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:10.727072 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:10.727098 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:10.728411 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:10.728869 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:10.728872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:10.728873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:10.728949 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:10.729257 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:10.729260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:10.729261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:10.729434 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:10.729565 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:10.729841 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 18:34:10.729931 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":45}
I20260806 18:34:10.745517 6576 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:10.746474 6576 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:10.751085 6576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e3dfd80c7154cd9a3d62bf7ac69c3f3: Generated new TSK 0
I20260806 18:34:10.751789 6576 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:10.760708 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 6466
W20260806 18:34:10.776160 6447 connection.cc:535] client connection to 127.7.67.254:36217 recv error: Network error: failed to read from TLS socket (remote: 127.7.67.254:36217): Connection reset by peer (error 104)
2026-08-06T18:34:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1504 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:10 dist-test-slave-5bn2 krb5kdc[6611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:10 dist-test-slave-5bn2 krb5kdc[6611](info): set up 2 sockets
Aug 06 18:34:10 dist-test-slave-5bn2 krb5kdc[6611](info): commencing operation
krb5kdc: starting...
W20260806 18:34:11.876477 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:34:11 dist-test-slave-5bn2 krb5kdc[6611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:34:11.999773 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:44475
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:35087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:44475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:12.115998 6626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:12.116283 6626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:12.116340 6626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:12.120378 6626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:12.120452 6626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:12.120473 6626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:12.120491 6626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:12.120512 6626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:12.125855 6626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:35087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:44475
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:44475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:12.126942 6626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:12.127974 6626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:12.136530 6640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:12.136547 6638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:12.136739 6636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:12.136752 6626 server_base.cc:1034] running on GCE node
I20260806 18:34:12.137493 6626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:12.138686 6626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:12.139878 6626 hybrid_clock.cc:648] HybridClock initialized: now 1786041252139864 us; error 41 us; skew 500 ppm
Aug 06 18:34:12 dist-test-slave-5bn2 krb5kdc[6611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041252, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:12.146440 6626 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:34:12.147466 6626 webserver.cc:460] Webserver started at http://127.7.67.254:43979/ using document root <none> and password file <none>
I20260806 18:34:12.148042 6626 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:12.148113 6626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:12.148330 6626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:12.150449 6626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "1d103abfabfb4deb9a74eb99aee5bbfb"
format_stamp: "Formatted at 2026-08-06 18:34:12 on dist-test-slave-5bn2"
server_key: "390d633ffa2ad9d1eef07cc834e9278a"
server_key_iv: "d1f30c0bd85a25c100b7e29114cec55a"
server_key_version: "encryptionkey@0"
I20260806 18:34:12.151038 6626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "1d103abfabfb4deb9a74eb99aee5bbfb"
format_stamp: "Formatted at 2026-08-06 18:34:12 on dist-test-slave-5bn2"
server_key: "390d633ffa2ad9d1eef07cc834e9278a"
server_key_iv: "d1f30c0bd85a25c100b7e29114cec55a"
server_key_version: "encryptionkey@0"
I20260806 18:34:12.154736 6626 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:12.157222 6651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:12.158460 6626 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:12.158602 6626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "1d103abfabfb4deb9a74eb99aee5bbfb"
format_stamp: "Formatted at 2026-08-06 18:34:12 on dist-test-slave-5bn2"
server_key: "390d633ffa2ad9d1eef07cc834e9278a"
server_key_iv: "d1f30c0bd85a25c100b7e29114cec55a"
server_key_version: "encryptionkey@0"
I20260806 18:34:12.158705 6626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:12.188109 6626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:12.190963 6626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:12.191147 6626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:12.197988 6626 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:44475
I20260806 18:34:12.198009 6721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:44475 every 8 connection(s)
I20260806 18:34:12.198938 6626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:12.201663 6723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:12.207006 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 6626
I20260806 18:34:12.207152 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:34:12.207273 6723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb: Bootstrap starting.
I20260806 18:34:12.207401 7439 external_mini_cluster.cc:1428] Setting key 13274915d000f3fbc4da56e21ec30da0
I20260806 18:34:12.209766 6723 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:12.210548 6723 log.cc:824] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:12.212594 6723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb: No bootstrap required, opened a new log
I20260806 18:34:12.216118 6731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.208713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52859 (local address 127.7.67.254:44475)
0806 18:34:12.209128 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:12.209144 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:12.209178 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:34:12.209793 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:12.209809 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:12.210186 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:12.210459 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:12.210821 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.211812 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.212964 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.213420 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.213529 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.214409 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:12.214414 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:12.214426 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:12.214787 (+ 361us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:12.214917 (+ 130us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:12.214947 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:12.215047 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:12.215260 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 18:34:12.215384 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":118,"threads_started":1}
I20260806 18:34:12.216549 6723 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d103abfabfb4deb9a74eb99aee5bbfb" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44475 } }
I20260806 18:34:12.216769 6723 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:12.216825 6723 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d103abfabfb4deb9a74eb99aee5bbfb, State: Initialized, Role: FOLLOWER
I20260806 18:34:12.217275 6723 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d103abfabfb4deb9a74eb99aee5bbfb" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44475 } }
I20260806 18:34:12.217449 6723 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:12.217525 6723 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:12.217621 6723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:12.218634 6723 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d103abfabfb4deb9a74eb99aee5bbfb" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44475 } }
I20260806 18:34:12.219060 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:12.219010 6723 leader_election.cc:302] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d103abfabfb4deb9a74eb99aee5bbfb; no voters:
I20260806 18:34:12.219317 6723 leader_election.cc:288] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:12.219406 6734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:12.219604 6734 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb [term 1 LEADER]: Becoming Leader. State: Replica: 1d103abfabfb4deb9a74eb99aee5bbfb, State: Running, Role: LEADER
I20260806 18:34:12.219981 6734 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d103abfabfb4deb9a74eb99aee5bbfb" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44475 } }
I20260806 18:34:12.220547 6723 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:12.221587 6735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d103abfabfb4deb9a74eb99aee5bbfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d103abfabfb4deb9a74eb99aee5bbfb" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44475 } } }
I20260806 18:34:12.221710 6735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:12.221946 6735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d103abfabfb4deb9a74eb99aee5bbfb. Latest consensus state: current_term: 1 leader_uuid: "1d103abfabfb4deb9a74eb99aee5bbfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d103abfabfb4deb9a74eb99aee5bbfb" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 44475 } } }
I20260806 18:34:12.222020 6735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:12.222132 6753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:12.225394 6753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:12.227222 6731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.221225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50235 (local address 127.7.67.254:44475)
0806 18:34:12.221333 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:12.221338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:12.222778 (+ 1440us) server_negotiation.cc:407] Connection header received
0806 18:34:12.222990 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:12.223009 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:12.223053 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:12.223128 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:12.223433 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.224150 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.225233 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.225566 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.226225 (+ 659us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.226742 (+ 517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:12.226746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:12.226749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:12.226762 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:12.226792 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:12.226806 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:12.226933 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:12.227051 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 18:34:12.227086 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:34:12.231420 6753 catalog_manager.cc:1347] Generated new cluster ID: a114e916892044398844931b42bcd9d8
I20260806 18:34:12.231495 6753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:12.245817 6753 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:12.246748 6753 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:12.262264 6753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d103abfabfb4deb9a74eb99aee5bbfb: Generated new TSK 0
I20260806 18:34:12.263018 6753 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:12.272866 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 6626
W20260806 18:34:12.285660 6607 connection.cc:535] client connection to 127.7.67.254:44475 recv error: Network error: failed to read from TLS socket (remote: 127.7.67.254:44475): Connection reset by peer (error 104)
2026-08-06T18:34:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1509 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:12 dist-test-slave-5bn2 krb5kdc[6769](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:12 dist-test-slave-5bn2 krb5kdc[6769](info): set up 2 sockets
Aug 06 18:34:12 dist-test-slave-5bn2 krb5kdc[6769](info): commencing operation
krb5kdc: starting...
W20260806 18:34:13.388983 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:34:13 dist-test-slave-5bn2 krb5kdc[6769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:34:13.513453 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:35767
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:36711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:35767
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:13.625056 6789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:13.625322 6789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:13.625378 6789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:13.629324 6789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:13.629384 6789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:13.629416 6789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:13.629439 6789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:13.629467 6789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:13.634629 6789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:36711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:35767
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:35767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:13.635689 6789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:13.636683 6789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:13.645043 6801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:13.644990 6804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:13.645031 6789 server_base.cc:1034] running on GCE node
W20260806 18:34:13.645000 6800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:13.645787 6789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:13.646989 6789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:13.648162 6789 hybrid_clock.cc:648] HybridClock initialized: now 1786041253648144 us; error 35 us; skew 500 ppm
Aug 06 18:34:13 dist-test-slave-5bn2 krb5kdc[6769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041253, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:13.651015 6789 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:34:13.651952 6789 webserver.cc:460] Webserver started at http://127.7.67.254:33303/ using document root <none> and password file <none>
I20260806 18:34:13.652539 6789 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:13.652609 6789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:13.652814 6789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:13.654954 6789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "f476f259583f426f9833064c699b19f5"
format_stamp: "Formatted at 2026-08-06 18:34:13 on dist-test-slave-5bn2"
server_key: "2f81f126d69d7f4e653ceebc70f2ef01"
server_key_iv: "e06d7d58c19d106e998bee9534f6f24e"
server_key_version: "encryptionkey@0"
I20260806 18:34:13.655524 6789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "f476f259583f426f9833064c699b19f5"
format_stamp: "Formatted at 2026-08-06 18:34:13 on dist-test-slave-5bn2"
server_key: "2f81f126d69d7f4e653ceebc70f2ef01"
server_key_iv: "e06d7d58c19d106e998bee9534f6f24e"
server_key_version: "encryptionkey@0"
I20260806 18:34:13.659232 6789 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:34:13.661592 6815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:13.662931 6789 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:13.663046 6789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "f476f259583f426f9833064c699b19f5"
format_stamp: "Formatted at 2026-08-06 18:34:13 on dist-test-slave-5bn2"
server_key: "2f81f126d69d7f4e653ceebc70f2ef01"
server_key_iv: "e06d7d58c19d106e998bee9534f6f24e"
server_key_version: "encryptionkey@0"
I20260806 18:34:13.663138 6789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:13.668599 6789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:13.671344 6789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:13.671507 6789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:13.678310 6789 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:35767
I20260806 18:34:13.678323 6886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:35767 every 8 connection(s)
I20260806 18:34:13.679270 6789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:13.682031 6887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:13.687520 6887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5: Bootstrap starting.
I20260806 18:34:13.689489 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 6789
I20260806 18:34:13.689642 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:34:13.689939 7439 external_mini_cluster.cc:1428] Setting key 05abdb0cfcb755644f16c4965ad8c52b
I20260806 18:34:13.690140 6887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:13.691023 6887 log.cc:824] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:13.692942 6887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5: No bootstrap required, opened a new log
I20260806 18:34:13.694811 6892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:13.691149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48605 (local address 127.7.67.254:35767)
0806 18:34:13.691590 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:13.691606 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:13.691641 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:34:13.692277 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:13.692294 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:13.692563 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:13.692835 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:13.693652 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:13.693657 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:13.693668 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:13.693678 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:13.693730 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:13.693749 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:13.693832 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:13.693956 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 18:34:13.694099 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":126,"threads_started":1}
I20260806 18:34:13.696903 6887 raft_consensus.cc:348] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f476f259583f426f9833064c699b19f5" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 35767 } }
I20260806 18:34:13.697162 6887 raft_consensus.cc:374] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:13.697240 6887 raft_consensus.cc:729] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f476f259583f426f9833064c699b19f5, State: Initialized, Role: FOLLOWER
I20260806 18:34:13.697656 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:13.697708 6887 consensus_queue.cc:260] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f476f259583f426f9833064c699b19f5" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 35767 } }
I20260806 18:34:13.697857 6887 raft_consensus.cc:388] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:13.697912 6887 raft_consensus.cc:482] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:13.697999 6887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:13.699142 6887 raft_consensus.cc:504] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f476f259583f426f9833064c699b19f5" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 35767 } }
I20260806 18:34:13.699509 6887 leader_election.cc:302] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f476f259583f426f9833064c699b19f5; no voters:
I20260806 18:34:13.699867 6887 leader_election.cc:288] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:13.700146 6899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:13.700390 6899 raft_consensus.cc:686] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5 [term 1 LEADER]: Becoming Leader. State: Replica: f476f259583f426f9833064c699b19f5, State: Running, Role: LEADER
I20260806 18:34:13.700762 6899 consensus_queue.cc:237] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f476f259583f426f9833064c699b19f5" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 35767 } }
I20260806 18:34:13.701180 6887 sys_catalog.cc:564] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:13.701809 6892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:13.699915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54523 (local address 127.7.67.254:35767)
0806 18:34:13.700024 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:13.700028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:13.700210 (+ 182us) server_negotiation.cc:407] Connection header received
0806 18:34:13.700375 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:13.700379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:13.700415 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:13.700495 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:13.701178 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:13.701182 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:13.701184 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:13.701197 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:13.701226 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:13.701239 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:13.701314 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:13.701658 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 18:34:13.701696 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:34:13.702822 6901 sys_catalog.cc:455] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f476f259583f426f9833064c699b19f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f476f259583f426f9833064c699b19f5" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 35767 } } }
I20260806 18:34:13.702891 6902 sys_catalog.cc:455] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f476f259583f426f9833064c699b19f5. Latest consensus state: current_term: 1 leader_uuid: "f476f259583f426f9833064c699b19f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f476f259583f426f9833064c699b19f5" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 35767 } } }
I20260806 18:34:13.702970 6901 sys_catalog.cc:458] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:13.703002 6902 sys_catalog.cc:458] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5 [sys.catalog]: This master's current role is: LEADER
W20260806 18:34:13.706202 6920 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:34:13.706281 6920 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:34:13.706336 6921 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:13.707201 6921 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:13.712536 6921 catalog_manager.cc:1347] Generated new cluster ID: 667d818e33a14f11a19ee5fab5d542a6
I20260806 18:34:13.712603 6921 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:13.722585 6921 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:13.723058 6921 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:13.731062 6921 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f476f259583f426f9833064c699b19f5: Generated new TSK 0
I20260806 18:34:13.731670 6921 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:13.734385 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 6789
2026-08-06T18:34:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1458 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:13 dist-test-slave-5bn2 krb5kdc[6927](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:13 dist-test-slave-5bn2 krb5kdc[6927](info): set up 2 sockets
Aug 06 18:34:13 dist-test-slave-5bn2 krb5kdc[6927](info): commencing operation
krb5kdc: starting...
W20260806 18:34:14.847244 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:34:14 dist-test-slave-5bn2 krb5kdc[6927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:34:14.967362 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:42477
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:33265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:42477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:15.079650 6951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:15.079916 6951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:15.079955 6951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:15.083917 6951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:15.083984 6951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:15.084003 6951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:15.084017 6951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:15.084030 6951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:15.089323 6951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:33265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:42477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:42477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:15.090373 6951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:15.091405 6951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:15.099696 6962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:15.099694 6960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:15.099696 6958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:15.100103 6951 server_base.cc:1034] running on GCE node
I20260806 18:34:15.100666 6951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:15.101749 6951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:15.102914 6951 hybrid_clock.cc:648] HybridClock initialized: now 1786041255102894 us; error 33 us; skew 500 ppm
Aug 06 18:34:15 dist-test-slave-5bn2 krb5kdc[6927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041255, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:15.105568 6951 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:34:15.106400 6951 webserver.cc:460] Webserver started at http://127.7.67.254:45761/ using document root <none> and password file <none>
I20260806 18:34:15.106945 6951 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:15.107017 6951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:15.107198 6951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:15.109215 6951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "38c2be69c34f484891e7740497c4ba0e"
format_stamp: "Formatted at 2026-08-06 18:34:15 on dist-test-slave-5bn2"
server_key: "e892bcc536509385e0fc4a2b5a928be6"
server_key_iv: "667f73eada5c3cefe397078dfed6ff22"
server_key_version: "encryptionkey@0"
I20260806 18:34:15.109743 6951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "38c2be69c34f484891e7740497c4ba0e"
format_stamp: "Formatted at 2026-08-06 18:34:15 on dist-test-slave-5bn2"
server_key: "e892bcc536509385e0fc4a2b5a928be6"
server_key_iv: "667f73eada5c3cefe397078dfed6ff22"
server_key_version: "encryptionkey@0"
I20260806 18:34:15.113571 6951 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:34:15.115968 6974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:15.117174 6951 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:34:15.117280 6951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "38c2be69c34f484891e7740497c4ba0e"
format_stamp: "Formatted at 2026-08-06 18:34:15 on dist-test-slave-5bn2"
server_key: "e892bcc536509385e0fc4a2b5a928be6"
server_key_iv: "667f73eada5c3cefe397078dfed6ff22"
server_key_version: "encryptionkey@0"
I20260806 18:34:15.117365 6951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:15.127184 6951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:15.130182 6951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:15.130331 6951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:15.137063 6951 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:42477
I20260806 18:34:15.137105 7055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:42477 every 8 connection(s)
I20260806 18:34:15.138003 6951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:15.140746 7056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:15.143622 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 6951
I20260806 18:34:15.143744 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:34:15.143997 7439 external_mini_cluster.cc:1428] Setting key c2b896ef1c7ab9afcad6600170b8a1cc
I20260806 18:34:15.146708 7056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e: Bootstrap starting.
I20260806 18:34:15.149389 7056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:15.150218 7056 log.cc:824] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:15.152309 7056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e: No bootstrap required, opened a new log
Aug 06 18:34:15 dist-test-slave-5bn2 krb5kdc[6927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:34:15.155985 7056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c2be69c34f484891e7740497c4ba0e" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 42477 } }
I20260806 18:34:15.156270 7056 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:15.156333 7056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c2be69c34f484891e7740497c4ba0e, State: Initialized, Role: FOLLOWER
I20260806 18:34:15.156829 7056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c2be69c34f484891e7740497c4ba0e" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 42477 } }
I20260806 18:34:15.156991 7056 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:15.157063 7056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:15.157171 7056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:15.158305 7056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c2be69c34f484891e7740497c4ba0e" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 42477 } }
I20260806 18:34:15.158666 7056 leader_election.cc:302] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38c2be69c34f484891e7740497c4ba0e; no voters:
I20260806 18:34:15.159073 7056 leader_election.cc:288] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:15.159191 7062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:15.159369 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.145363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57817 (local address 127.7.67.254:42477)
0806 18:34:15.145820 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.145839 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.145883 (+ 44us) server_negotiation.cc:407] Connection header received
0806 18:34:15.146672 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.146693 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.147140 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.147502 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:15.148152 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.149635 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.150510 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.151090 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.151243 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.153615 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:15.153621 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:15.153638 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:15.153670 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:15.156151 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:15.156614 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:15.156620 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:15.156627 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:15.156764 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:15.157013 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:15.157017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:15.157019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:15.157597 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:15.157791 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.158016 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.158334 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":138,"threads_started":1}
I20260806 18:34:15.159387 7062 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e [term 1 LEADER]: Becoming Leader. State: Replica: 38c2be69c34f484891e7740497c4ba0e, State: Running, Role: LEADER
I20260806 18:34:15.159752 7062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c2be69c34f484891e7740497c4ba0e" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 42477 } }
I20260806 18:34:15.160312 7056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:15.162645 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:15.163040 7064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38c2be69c34f484891e7740497c4ba0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c2be69c34f484891e7740497c4ba0e" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 42477 } } }
I20260806 18:34:15.163483 7064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:15.163079 7063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38c2be69c34f484891e7740497c4ba0e. Latest consensus state: current_term: 1 leader_uuid: "38c2be69c34f484891e7740497c4ba0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c2be69c34f484891e7740497c4ba0e" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 42477 } } }
I20260806 18:34:15.163761 7063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e [sys.catalog]: This master's current role is: LEADER
W20260806 18:34:15.165666 7084 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:34:15.165747 7084 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:34:15.165805 7086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:15.166661 7086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:15.172339 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.164755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34205 (local address 127.7.67.254:42477)
0806 18:34:15.164859 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.164863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.165071 (+ 208us) server_negotiation.cc:407] Connection header received
0806 18:34:15.165170 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.165172 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.165214 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.165276 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:15.165940 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.166487 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.167372 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.167614 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.167702 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.169086 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:15.169090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:15.169093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:15.169118 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:15.170577 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:15.171011 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:15.171015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:15.171017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:15.171109 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:15.171400 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:15.171404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:15.171405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:15.171675 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:15.171806 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.172059 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.172180 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:34:15.175439 7086 catalog_manager.cc:1347] Generated new cluster ID: 7f5300f335754af989206184a4b1a165
I20260806 18:34:15.175503 7086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:15.183578 7086 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:15.184474 7086 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:15.205209 7086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38c2be69c34f484891e7740497c4ba0e: Generated new TSK 0
I20260806 18:34:15.206097 7086 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:15.217564 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 6951
2026-08-06T18:34:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1489 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:15 dist-test-slave-5bn2 krb5kdc[7102](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:15 dist-test-slave-5bn2 krb5kdc[7102](info): set up 2 sockets
Aug 06 18:34:15 dist-test-slave-5bn2 krb5kdc[7102](info): commencing operation
krb5kdc: starting...
W20260806 18:34:16.331494 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:34:16 dist-test-slave-5bn2 krb5kdc[7102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:34:16.456035 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:45473
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:38151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:45473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:16.568542 7124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:16.568835 7124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:16.568895 7124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:16.572932 7124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:16.572996 7124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:16.573022 7124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:16.573043 7124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:16.573068 7124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:16.578414 7124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:38151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:45473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:45473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:16.579591 7124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:16.580664 7124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:16.589103 7135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:16.589193 7133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:16.589092 7132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:16.589349 7124 server_base.cc:1034] running on GCE node
I20260806 18:34:16.590000 7124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:16.591332 7124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:16.592510 7124 hybrid_clock.cc:648] HybridClock initialized: now 1786041256592496 us; error 35 us; skew 500 ppm
Aug 06 18:34:16 dist-test-slave-5bn2 krb5kdc[7102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041256, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:16.595419 7124 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:34:16.596321 7124 webserver.cc:460] Webserver started at http://127.7.67.254:39249/ using document root <none> and password file <none>
I20260806 18:34:16.596868 7124 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:16.596943 7124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:16.597164 7124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:16.599327 7124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "6c6837d15ef2422dade3d124c99186f0"
format_stamp: "Formatted at 2026-08-06 18:34:16 on dist-test-slave-5bn2"
server_key: "9b3710db8d793329227fb122f8af7cf0"
server_key_iv: "0d8113b17970242981166333bb1e51c6"
server_key_version: "encryptionkey@0"
I20260806 18:34:16.599917 7124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "6c6837d15ef2422dade3d124c99186f0"
format_stamp: "Formatted at 2026-08-06 18:34:16 on dist-test-slave-5bn2"
server_key: "9b3710db8d793329227fb122f8af7cf0"
server_key_iv: "0d8113b17970242981166333bb1e51c6"
server_key_version: "encryptionkey@0"
I20260806 18:34:16.603706 7124 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:34:16.606175 7151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:16.607409 7124 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:16.607530 7124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "6c6837d15ef2422dade3d124c99186f0"
format_stamp: "Formatted at 2026-08-06 18:34:16 on dist-test-slave-5bn2"
server_key: "9b3710db8d793329227fb122f8af7cf0"
server_key_iv: "0d8113b17970242981166333bb1e51c6"
server_key_version: "encryptionkey@0"
I20260806 18:34:16.607630 7124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:16.621821 7124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:16.624992 7124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:16.625180 7124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:16.632169 7124 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:45473
I20260806 18:34:16.632233 7224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:45473 every 8 connection(s)
I20260806 18:34:16.633106 7124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:16.635933 7225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:16.641430 7225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0: Bootstrap starting.
I20260806 18:34:16.642925 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 7124
I20260806 18:34:16.643056 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:34:16.643311 7439 external_mini_cluster.cc:1428] Setting key b11d3af1a753190308559b08d28556da
I20260806 18:34:16.644050 7225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:16.644964 7225 log.cc:824] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:16.646975 7225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0: No bootstrap required, opened a new log
I20260806 18:34:16.651262 7225 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6837d15ef2422dade3d124c99186f0" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45473 } }
I20260806 18:34:16.651562 7225 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:16.651623 7225 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6837d15ef2422dade3d124c99186f0, State: Initialized, Role: FOLLOWER
I20260806 18:34:16.652081 7225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6837d15ef2422dade3d124c99186f0" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45473 } }
I20260806 18:34:16.652215 7225 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:16.652276 7225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:16.652333 7229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:16.644408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42027 (local address 127.7.67.254:45473)
0806 18:34:16.644827 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:16.644844 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:16.644879 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:34:16.645532 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:16.645550 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:16.645885 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:16.646170 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:16.646558 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:16.647695 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:16.648951 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:16.649444 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:16.649567 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:16.650451 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:16.650457 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:16.650471 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:16.650914 (+ 443us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:16.650976 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:16.651001 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:16.651077 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:16.651225 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 18:34:16.651439 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":121,"threads_started":1}
I20260806 18:34:16.652371 7225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:16.653296 7225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6837d15ef2422dade3d124c99186f0" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45473 } }
I20260806 18:34:16.653630 7225 leader_election.cc:302] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c6837d15ef2422dade3d124c99186f0; no voters:
I20260806 18:34:16.653931 7225 leader_election.cc:288] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:16.654032 7232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:16.654255 7232 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0 [term 1 LEADER]: Becoming Leader. State: Replica: 6c6837d15ef2422dade3d124c99186f0, State: Running, Role: LEADER
I20260806 18:34:16.654611 7232 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6837d15ef2422dade3d124c99186f0" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45473 } }
I20260806 18:34:16.655285 7225 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:16.655303 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:16.655961 7232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c6837d15ef2422dade3d124c99186f0. Latest consensus state: current_term: 1 leader_uuid: "6c6837d15ef2422dade3d124c99186f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6837d15ef2422dade3d124c99186f0" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45473 } } }
I20260806 18:34:16.656180 7232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:16.656391 7235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c6837d15ef2422dade3d124c99186f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6837d15ef2422dade3d124c99186f0" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 45473 } } }
I20260806 18:34:16.656512 7235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:16.656606 7250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:16.660089 7250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:16.662170 7229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:16.657977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48005 (local address 127.7.67.254:45473)
0806 18:34:16.658087 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:16.658091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:16.658389 (+ 298us) server_negotiation.cc:407] Connection header received
0806 18:34:16.658596 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:16.658599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:16.658646 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:16.658717 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:16.659033 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:16.659545 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:16.660605 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:16.660931 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:16.661026 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:16.661783 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:16.661787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:16.661790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:16.661802 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:16.661830 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:16.661840 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:16.661897 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:16.661993 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 18:34:16.662042 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 18:34:16.665748 7250 catalog_manager.cc:1347] Generated new cluster ID: 8d2c8237ffe84fdb9686592c7d5419af
I20260806 18:34:16.665814 7250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:16.674296 7250 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:16.676231 7250 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:16.685083 7250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c6837d15ef2422dade3d124c99186f0: Generated new TSK 0
I20260806 18:34:16.685791 7250 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:16.691660 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 7124
2026-08-06T18:34:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1470 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:16 dist-test-slave-5bn2 krb5kdc[7274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:16 dist-test-slave-5bn2 krb5kdc[7274](info): set up 2 sockets
Aug 06 18:34:16 dist-test-slave-5bn2 krb5kdc[7274](info): commencing operation
krb5kdc: starting...
W20260806 18:34:17.809312 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:34:17 dist-test-slave-5bn2 krb5kdc[7274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:34:17.933244 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:43011
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:35691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:43011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:18.046515 7296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:18.046815 7296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:18.046902 7296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:18.050978 7296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:18.051046 7296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:18.051074 7296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:18.051095 7296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:18.051126 7296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:18.056448 7296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:35691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:43011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:43011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:18.057592 7296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:18.058655 7296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:18.067740 7302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:18.067746 7305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:18.067718 7301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:18.067989 7296 server_base.cc:1034] running on GCE node
I20260806 18:34:18.068648 7296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:18.069761 7296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:18.070920 7296 hybrid_clock.cc:648] HybridClock initialized: now 1786041258070898 us; error 38 us; skew 500 ppm
Aug 06 18:34:18 dist-test-slave-5bn2 krb5kdc[7274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041258, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:18.078779 7296 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:34:18.079833 7296 webserver.cc:460] Webserver started at http://127.7.67.254:44329/ using document root <none> and password file <none>
I20260806 18:34:18.080410 7296 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:18.080476 7296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:18.080693 7296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:18.082836 7296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "fb888d768ddd44f99adc51a08320cf2b"
format_stamp: "Formatted at 2026-08-06 18:34:18 on dist-test-slave-5bn2"
server_key: "99d0952fbab2b2989bf6f6769a8031f9"
server_key_iv: "3c60d13c55229bbe9f97c3e01ea2d0aa"
server_key_version: "encryptionkey@0"
I20260806 18:34:18.083460 7296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "fb888d768ddd44f99adc51a08320cf2b"
format_stamp: "Formatted at 2026-08-06 18:34:18 on dist-test-slave-5bn2"
server_key: "99d0952fbab2b2989bf6f6769a8031f9"
server_key_iv: "3c60d13c55229bbe9f97c3e01ea2d0aa"
server_key_version: "encryptionkey@0"
I20260806 18:34:18.087311 7296 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260806 18:34:18.089854 7312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:18.091193 7296 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:18.091325 7296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "fb888d768ddd44f99adc51a08320cf2b"
format_stamp: "Formatted at 2026-08-06 18:34:18 on dist-test-slave-5bn2"
server_key: "99d0952fbab2b2989bf6f6769a8031f9"
server_key_iv: "3c60d13c55229bbe9f97c3e01ea2d0aa"
server_key_version: "encryptionkey@0"
I20260806 18:34:18.091437 7296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:18.096122 7296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:18.099310 7296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:18.099483 7296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:18.106361 7296 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:43011
I20260806 18:34:18.106374 7391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:43011 every 8 connection(s)
I20260806 18:34:18.107363 7296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:18.110198 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 7296
I20260806 18:34:18.110260 7392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:18.110347 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:34:18.110675 7439 external_mini_cluster.cc:1428] Setting key b3fabf05909898b2b1dcdc5cb0aa1bd3
I20260806 18:34:18.116102 7395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:18.112057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48073 (local address 127.7.67.254:43011)
0806 18:34:18.112472 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:18.112488 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:18.112538 (+ 50us) server_negotiation.cc:407] Connection header received
0806 18:34:18.113425 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:18.113443 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:18.113744 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:18.114027 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:18.114897 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:18.114902 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:18.114915 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:18.114927 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:18.114985 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:18.115006 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:18.115126 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:18.115224 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 18:34:18.115368 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":112,"threads_started":1}
I20260806 18:34:18.116887 7392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b: Bootstrap starting.
I20260806 18:34:18.118968 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:18.119575 7392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:18.120499 7392 log.cc:824] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:18.122905 7392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b: No bootstrap required, opened a new log
I20260806 18:34:18.122915 7395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:18.121320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54765 (local address 127.7.67.254:43011)
0806 18:34:18.121437 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:18.121442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:18.121650 (+ 208us) server_negotiation.cc:407] Connection header received
0806 18:34:18.121760 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:18.121762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:18.121792 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:18.121854 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:18.122531 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:18.122534 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:18.122536 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:18.122546 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:18.122572 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:18.122582 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:18.122642 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:18.122747 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 18:34:18.122779 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:34:18.126678 7392 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb888d768ddd44f99adc51a08320cf2b" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 43011 } }
I20260806 18:34:18.126947 7392 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:18.127010 7392 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb888d768ddd44f99adc51a08320cf2b, State: Initialized, Role: FOLLOWER
I20260806 18:34:18.127417 7392 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb888d768ddd44f99adc51a08320cf2b" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 43011 } }
I20260806 18:34:18.127593 7392 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:18.127657 7392 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:18.127758 7392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:18.128674 7392 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb888d768ddd44f99adc51a08320cf2b" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 43011 } }
I20260806 18:34:18.128995 7392 leader_election.cc:302] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb888d768ddd44f99adc51a08320cf2b; no voters:
I20260806 18:34:18.129312 7392 leader_election.cc:288] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:18.129401 7404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:18.129616 7404 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b [term 1 LEADER]: Becoming Leader. State: Replica: fb888d768ddd44f99adc51a08320cf2b, State: Running, Role: LEADER
I20260806 18:34:18.129988 7404 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb888d768ddd44f99adc51a08320cf2b" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 43011 } }
I20260806 18:34:18.130515 7392 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:18.131587 7407 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb888d768ddd44f99adc51a08320cf2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb888d768ddd44f99adc51a08320cf2b" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 43011 } } }
I20260806 18:34:18.131714 7407 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:18.132042 7408 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb888d768ddd44f99adc51a08320cf2b. Latest consensus state: current_term: 1 leader_uuid: "fb888d768ddd44f99adc51a08320cf2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb888d768ddd44f99adc51a08320cf2b" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 43011 } } }
I20260806 18:34:18.132125 7408 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:18.132493 7419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:18.135175 7419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:18.140297 7419 catalog_manager.cc:1347] Generated new cluster ID: 00c3a0c6d584470e9d0b3f7591fe6088
I20260806 18:34:18.140367 7419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:18.159737 7419 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:18.160113 7419 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:18.171202 7419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb888d768ddd44f99adc51a08320cf2b: Generated new TSK 0
I20260806 18:34:18.172094 7419 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:18.175958 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 7296
2026-08-06T18:34:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1492 ms)
[----------] 18 tests from AuthTokenIssuingTest (26973 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:18 dist-test-slave-5bn2 krb5kdc[7435](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:18 dist-test-slave-5bn2 krb5kdc[7435](info): set up 2 sockets
Aug 06 18:34:18 dist-test-slave-5bn2 krb5kdc[7435](info): commencing operation
krb5kdc: starting...
W20260806 18:34:19.305653 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:34:19 dist-test-slave-5bn2 krb5kdc[7435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:34:19.431155 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:40895
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:41767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:40895,127.7.67.253:39215,127.7.67.252:42259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:19.543582 7461 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:34:19.543866 7461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:19.543888 7461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:19.543924 7461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:19.547832 7461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:19.547890 7461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:19.547909 7461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:19.547923 7461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:19.547936 7461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:19.553126 7461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:41767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:40895,127.7.67.253:39215,127.7.67.252:42259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:40895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:19.554117 7461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:19.555142 7461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:19.563349 7467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:19.563365 7461 server_base.cc:1034] running on GCE node
W20260806 18:34:19.563369 7468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:19.563462 7470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:19.564143 7461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:19.565199 7461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:19.566360 7461 hybrid_clock.cc:648] HybridClock initialized: now 1786041259566346 us; error 28 us; skew 500 ppm
Aug 06 18:34:19 dist-test-slave-5bn2 krb5kdc[7435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041259, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:19.569028 7461 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:34:19.569859 7461 webserver.cc:460] Webserver started at http://127.7.67.254:38265/ using document root <none> and password file <none>
I20260806 18:34:19.570385 7461 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:19.570451 7461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:19.570622 7461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:19.572674 7461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "bc92dd2874cc4ed28316da4650831578"
format_stamp: "Formatted at 2026-08-06 18:34:19 on dist-test-slave-5bn2"
server_key: "6fe70976ff3f77fa7a0e4cf0f7fbf387"
server_key_iv: "154efe1b70b532b8572278ad10180266"
server_key_version: "encryptionkey@0"
I20260806 18:34:19.573194 7461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "bc92dd2874cc4ed28316da4650831578"
format_stamp: "Formatted at 2026-08-06 18:34:19 on dist-test-slave-5bn2"
server_key: "6fe70976ff3f77fa7a0e4cf0f7fbf387"
server_key_iv: "154efe1b70b532b8572278ad10180266"
server_key_version: "encryptionkey@0"
I20260806 18:34:19.576809 7461 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:34:19.579135 7477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:19.580302 7461 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:34:19.580413 7461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "bc92dd2874cc4ed28316da4650831578"
format_stamp: "Formatted at 2026-08-06 18:34:19 on dist-test-slave-5bn2"
server_key: "6fe70976ff3f77fa7a0e4cf0f7fbf387"
server_key_iv: "154efe1b70b532b8572278ad10180266"
server_key_version: "encryptionkey@0"
I20260806 18:34:19.580500 7461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:19.612306 7461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:19.615213 7461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:19.615393 7461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:19.622360 7461 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:40895
I20260806 18:34:19.622628 7549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:40895 every 8 connection(s)
I20260806 18:34:19.623389 7461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:19.626157 7550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:19.628075 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 7461
I20260806 18:34:19.628221 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:34:19.628492 7439 external_mini_cluster.cc:1428] Setting key 45cd235cd5155dd0502466daddd1d9ad
I20260806 18:34:19.630405 7550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.67.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.253@KRBTEST.COM" created.
Aug 06 18:34:19 dist-test-slave-5bn2 krb5kdc[7435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041259, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:34:19.645812 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.632785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:40895 (local address 127.0.0.1:38306)
0806 18:34:19.633420 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.633470 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:19.634313 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:19.635113 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:19.635118 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:19.635458 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:19.635802 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:19.635817 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:19.637431 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:19.637433 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:19.638197 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:19.638207 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:19.638828 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:19.638831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:19.638967 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:19.639483 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:19.639504 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:19.641025 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:19.642832 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:19.642842 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:19.642871 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:19.643176 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:19.643487 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:19.643490 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:19.643492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:19.643608 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:19.644175 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:19.644181 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:19.644316 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 18:34:19.644467 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 18:34:19.644836 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":115,"spinlock_wait_cycles":55040,"thread_start_us":174,"threads_started":1}
I20260806 18:34:19.645812 7553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.632784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38306 (local address 127.7.67.254:40895)
0806 18:34:19.633302 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:19.633321 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:19.633587 (+ 266us) server_negotiation.cc:407] Connection header received
0806 18:34:19.634631 (+ 1044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:19.634638 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:19.634911 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:19.635124 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:19.635945 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:19.637309 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:19.638304 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:19.638696 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:19.638814 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:19.641161 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:19.641167 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:19.641177 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:19.641197 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:19.642719 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:19.643277 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:19.643282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:19.643283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:19.643391 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:19.643695 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:19.643698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:19.643699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:19.644066 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:19.644187 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:19.644544 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 18:34:19.644774 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":344,"thread_start_us":153,"threads_started":1}
I20260806 18:34:19.648474 7550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:34:19.649281 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.648937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:39215 (local address 127.0.0.1:37462)
0806 18:34:19.649109 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.649196 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.253:39215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260806 18:34:19.649538 7483 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.67.253:39215: connect: Connection refused (error 111)
I20260806 18:34:19.650455 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.650223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:39215 (local address 127.0.0.1:37468)
0806 18:34:19.650340 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.650394 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.253:39215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 18:34:19.650581 7550 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.253:39215: Network error: Client connection negotiation failed: client connection to 127.7.67.253:39215: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.253.keytab.
Entry for principal kudu/127.7.67.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.253.keytab.
I20260806 18:34:19.657166 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.253:39215
--webserver_interface=127.7.67.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.253.keytab
--principal=kudu/127.7.67.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:41767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:40895,127.7.67.253:39215,127.7.67.252:42259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:34:19.699817 7550 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } attempt: 1
I20260806 18:34:19.700659 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.700365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:39215 (local address 127.0.0.1:37478)
0806 18:34:19.700511 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.700577 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.253:39215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 18:34:19.701454 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.701263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:39215 (local address 127.0.0.1:37492)
0806 18:34:19.701351 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.701396 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.253:39215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 18:34:19.701596 7550 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.253:39215: Network error: Client connection negotiation failed: client connection to 127.7.67.253:39215: connect: Connection refused (error 111)
I20260806 18:34:19.769749 7550 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } attempt: 2
I20260806 18:34:19.770651 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.770349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:39215 (local address 127.0.0.1:37494)
0806 18:34:19.770505 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.770568 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.253:39215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 18:34:19.771314 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.771137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:39215 (local address 127.0.0.1:37504)
0806 18:34:19.771218 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.771260 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.253:39215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 18:34:19.771420 7550 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.253:39215: Network error: Client connection negotiation failed: client connection to 127.7.67.253:39215: connect: Connection refused (error 111)
W20260806 18:34:19.771735 7559 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:34:19.772014 7559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:19.772042 7559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:19.772094 7559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:19.776065 7559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:19.776120 7559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:19.776137 7559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:19.776151 7559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:19.776171 7559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:19.781497 7559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:41767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:40895,127.7.67.253:39215,127.7.67.252:42259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.253.keytab
--principal=kudu/127.7.67.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.253:39215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.67.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:19.782531 7559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:19.783578 7559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:19.791867 7569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:19.791872 7565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:19.791893 7567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:19.792201 7559 server_base.cc:1034] running on GCE node
I20260806 18:34:19.792750 7559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:19.793802 7559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:19.794917 7559 hybrid_clock.cc:648] HybridClock initialized: now 1786041259794903 us; error 33 us; skew 500 ppm
Aug 06 18:34:19 dist-test-slave-5bn2 krb5kdc[7435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041259, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:19.797529 7559 init.cc:377] Logged in from keytab as kudu/127.7.67.253@KRBTEST.COM (short username kudu)
I20260806 18:34:19.798388 7559 webserver.cc:460] Webserver started at http://127.7.67.253:37935/ using document root <none> and password file <none>
I20260806 18:34:19.798960 7559 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:19.799036 7559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:19.799240 7559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:19.801234 7559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-1/data/instance:
uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52"
format_stamp: "Formatted at 2026-08-06 18:34:19 on dist-test-slave-5bn2"
server_key: "85f8c81a1df9fbb05d31a3303dbdf4c3"
server_key_iv: "b0527815f84ae2acbf3360c7f04f3225"
server_key_version: "encryptionkey@0"
I20260806 18:34:19.801784 7559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-1/wal/instance:
uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52"
format_stamp: "Formatted at 2026-08-06 18:34:19 on dist-test-slave-5bn2"
server_key: "85f8c81a1df9fbb05d31a3303dbdf4c3"
server_key_iv: "b0527815f84ae2acbf3360c7f04f3225"
server_key_version: "encryptionkey@0"
I20260806 18:34:19.805688 7559 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:34:19.808676 7579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:19.809875 7559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:34:19.810005 7559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-1/wal
uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52"
format_stamp: "Formatted at 2026-08-06 18:34:19 on dist-test-slave-5bn2"
server_key: "85f8c81a1df9fbb05d31a3303dbdf4c3"
server_key_iv: "b0527815f84ae2acbf3360c7f04f3225"
server_key_version: "encryptionkey@0"
I20260806 18:34:19.810119 7559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:19.838281 7559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:19.841174 7559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:19.841361 7559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:19.848369 7559 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.253:39215
I20260806 18:34:19.848626 7657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.253:39215 every 8 connection(s)
I20260806 18:34:19.849355 7559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-1/data/info.pb
I20260806 18:34:19.852227 7658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:19.854034 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 7559
I20260806 18:34:19.854159 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-1/wal/instance
I20260806 18:34:19.854444 7439 external_mini_cluster.cc:1428] Setting key afd2e23037d3d19a771b891a1797dee9
I20260806 18:34:19.856817 7658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:34:19.862617 7550 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } attempt: 3
WARNING: no policy specified for kudu/127.7.67.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.252@KRBTEST.COM" created.
Aug 06 18:34:19 dist-test-slave-5bn2 krb5kdc[7435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041259, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.253@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
Aug 06 18:34:19 dist-test-slave-5bn2 krb5kdc[7435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041259, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for kudu/127.7.67.253@KRBTEST.COM
I20260806 18:34:19.872453 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.863099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:39215 (local address 127.0.0.1:37516)
0806 18:34:19.863236 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.863251 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:19.863328 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:19.863995 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:19.863999 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:19.864182 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:19.864295 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:19.864302 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:19.865532 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:19.865536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:19.866117 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:19.866127 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:19.866875 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:19.866879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:19.866976 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:19.867414 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:19.867426 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:19.868745 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:19.870824 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:19.870827 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:19.870829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:19.871083 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:19.871394 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:19.871397 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:19.871399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:19.871479 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:19.871926 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:19.871929 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:19.872044 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 18:34:19.872131 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 18:34:19.872270 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"spinlock_wait_cycles":768}
I20260806 18:34:19.872741 7553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.858747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38318 (local address 127.7.67.254:40895)
0806 18:34:19.858909 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:19.858913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:19.860010 (+ 1097us) server_negotiation.cc:407] Connection header received
0806 18:34:19.860929 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:19.860933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:19.861000 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:19.861072 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:19.862236 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:19.862779 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:19.864565 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:19.864916 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:19.865022 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:19.867730 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:19.867734 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:19.867736 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:19.867761 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:19.869663 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:19.870360 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:19.870364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:19.870366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:19.870452 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:19.870878 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:19.870881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:19.870882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:19.871090 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:19.871176 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:19.872073 (+ 897us) server_negotiation.cc:299] Negotiation successful
0806 18:34:19.872205 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46,"spinlock_wait_cycles":4992}
I20260806 18:34:19.873443 7662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.863175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37516 (local address 127.7.67.253:39215)
0806 18:34:19.863545 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:19.863557 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:19.863583 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:34:19.863625 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:19.863631 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:19.863828 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:19.863961 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:19.864435 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:19.865391 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:19.866252 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:19.866695 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:19.866837 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:19.868898 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:19.868905 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:19.868920 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:19.868945 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:19.870734 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:19.871202 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:19.871208 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:19.871210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:19.871299 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:19.871571 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:19.871574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:19.871576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:19.871823 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:19.871939 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:19.872243 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 18:34:19.872550 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":158,"threads_started":1}
I20260806 18:34:19.873955 7661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.859326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:40895 (local address 127.0.0.1:38318)
0806 18:34:19.859847 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.859892 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:19.860725 (+ 833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:19.861246 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:19.861253 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:19.861608 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:19.862092 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:19.862105 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:19.862919 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:19.862922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:19.864427 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:19.864440 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:19.865051 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:19.865053 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:19.865210 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:19.865776 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:19.865792 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:19.867587 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:19.869765 (+ 2178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:19.869776 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:19.869788 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:19.870231 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:19.870549 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:19.870552 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:19.870554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:19.870735 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:19.871418 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:19.871425 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:19.871760 (+ 335us) client_negotiation.cc:769] Sending connection context
0806 18:34:19.872314 (+ 554us) client_negotiation.cc:240] Negotiation successful
0806 18:34:19.873804 (+ 1490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":1268,"thread_start_us":149,"threads_started":1}
I20260806 18:34:19.875221 7658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:34:19.875466 7550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:34:19.876142 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.875871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:42259 (local address 127.0.0.1:54620)
0806 18:34:19.876031 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.876086 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:42259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 18:34:19.876804 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.876546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:42259 (local address 127.0.0.1:54636)
0806 18:34:19.876689 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.876741 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:42259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 18:34:19.876922 7550 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.252:42259: Network error: Client connection negotiation failed: client connection to 127.7.67.252:42259: connect: Connection refused (error 111)
Aug 06 18:34:19 dist-test-slave-5bn2 krb5kdc[7435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041259, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.253@KRBTEST.COM for kudu/127.7.67.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.252.keytab.
Entry for principal kudu/127.7.67.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.252.keytab.
I20260806 18:34:19.883711 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.252:42259
--webserver_interface=127.7.67.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.252.keytab
--principal=kudu/127.7.67.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:41767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:40895,127.7.67.253:39215,127.7.67.252:42259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:34:19.883924 7661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.875737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:39215 (local address 127.0.0.1:37526)
0806 18:34:19.875826 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.875840 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:19.875909 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:19.876119 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:19.876122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:19.876275 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:19.876368 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:19.876373 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:19.877245 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:19.877248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:19.877902 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:19.877912 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:19.878335 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:19.878338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:19.878451 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:19.878891 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:19.878905 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:19.880351 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:19.882252 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:19.882255 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:19.882257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:19.882429 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:19.882740 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:19.882742 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:19.882745 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:19.882824 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:19.883241 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:19.883245 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:19.883377 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 18:34:19.883566 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 18:34:19.883706 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":17}
I20260806 18:34:19.883941 7662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.875871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37526 (local address 127.7.67.253:39215)
0806 18:34:19.875952 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:19.875955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:19.875966 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:34:19.876003 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:19.876005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:19.876040 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:19.876097 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:19.876467 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:19.877152 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:19.878011 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:19.878244 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:19.878326 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:19.880503 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:19.880506 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:19.880509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:19.880534 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:19.882152 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:19.882530 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:19.882533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:19.882535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:19.882649 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:19.882932 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:19.882934 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:19.882936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:19.883144 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:19.883238 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:19.883619 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 18:34:19.883776 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":31}
I20260806 18:34:19.885839 7658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:34:19.886619 7661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.886253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:42259 (local address 127.0.0.1:54646)
0806 18:34:19.886455 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.886551 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:42259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
W20260806 18:34:19.887240 7582 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.67.252:42259: connect: Connection refused (error 111)
I20260806 18:34:19.888087 7661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.887882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:42259 (local address 127.0.0.1:54658)
0806 18:34:19.887974 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.888025 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:42259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 18:34:19.888222 7658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.252:42259: Network error: Client connection negotiation failed: client connection to 127.7.67.252:42259: connect: Connection refused (error 111)
I20260806 18:34:19.908061 7550 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } attempt: 1
I20260806 18:34:19.908948 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.908638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:42259 (local address 127.0.0.1:54666)
0806 18:34:19.908793 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.908868 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:42259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 18:34:19.909621 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.909430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:42259 (local address 127.0.0.1:54676)
0806 18:34:19.909524 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.909564 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:42259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 18:34:19.909729 7550 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.252:42259: Network error: Client connection negotiation failed: client connection to 127.7.67.252:42259: connect: Connection refused (error 111)
I20260806 18:34:19.937479 7658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } attempt: 1
I20260806 18:34:19.938337 7661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.938036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:42259 (local address 127.0.0.1:54692)
0806 18:34:19.938187 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.938252 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:42259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 18:34:19.939149 7661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.938939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:42259 (local address 127.0.0.1:54696)
0806 18:34:19.939040 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.939087 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:42259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 18:34:19.939265 7658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.252:42259: Network error: Client connection negotiation failed: client connection to 127.7.67.252:42259: connect: Connection refused (error 111)
I20260806 18:34:19.984859 7550 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } attempt: 2
I20260806 18:34:19.985700 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.985395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:42259 (local address 127.0.0.1:54710)
0806 18:34:19.985555 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.985619 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:42259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 18:34:19.986349 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.986153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:42259 (local address 127.0.0.1:54722)
0806 18:34:19.986249 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.986292 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:42259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 18:34:19.986460 7550 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.252:42259: Network error: Client connection negotiation failed: client connection to 127.7.67.252:42259: connect: Connection refused (error 111)
W20260806 18:34:19.999069 7668 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:34:19.999361 7668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:19.999403 7668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:19.999464 7668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:20.003379 7668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:20.003435 7668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:20.003461 7668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:20.003484 7668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:20.003512 7668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:20.007407 7658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } attempt: 2
I20260806 18:34:20.008150 7661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:20.007869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:42259 (local address 127.0.0.1:54738)
0806 18:34:20.008015 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:20.008074 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:42259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 18:34:20.008731 7668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:41767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:40895,127.7.67.253:39215,127.7.67.252:42259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.252.keytab
--principal=kudu/127.7.67.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.252:42259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.67.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:20.008814 7661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:20.008614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:42259 (local address 127.0.0.1:54750)
0806 18:34:20.008711 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:20.008755 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:42259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 18:34:20.008922 7658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.252:42259: Network error: Client connection negotiation failed: client connection to 127.7.67.252:42259: connect: Connection refused (error 111)
I20260806 18:34:20.009784 7668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:20.010805 7668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:20.018913 7681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:20.018905 7685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:20.018905 7678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:20.018983 7668 server_base.cc:1034] running on GCE node
I20260806 18:34:20.019726 7668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:20.020833 7668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:20.021989 7668 hybrid_clock.cc:648] HybridClock initialized: now 1786041260021971 us; error 31 us; skew 500 ppm
Aug 06 18:34:20 dist-test-slave-5bn2 krb5kdc[7435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:20.024696 7668 init.cc:377] Logged in from keytab as kudu/127.7.67.252@KRBTEST.COM (short username kudu)
I20260806 18:34:20.025558 7668 webserver.cc:460] Webserver started at http://127.7.67.252:38119/ using document root <none> and password file <none>
I20260806 18:34:20.026077 7668 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:20.026134 7668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:20.026305 7668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:20.028375 7668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-2/data/instance:
uuid: "bed777f10a204fbcb1f8c9724f03c7ae"
format_stamp: "Formatted at 2026-08-06 18:34:20 on dist-test-slave-5bn2"
server_key: "caa5e5d6f48d2f3da02e3e26d3207129"
server_key_iv: "bcaac80ed1aaa79bd132be06d007ce86"
server_key_version: "encryptionkey@0"
I20260806 18:34:20.028940 7668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-2/wal/instance:
uuid: "bed777f10a204fbcb1f8c9724f03c7ae"
format_stamp: "Formatted at 2026-08-06 18:34:20 on dist-test-slave-5bn2"
server_key: "caa5e5d6f48d2f3da02e3e26d3207129"
server_key_iv: "bcaac80ed1aaa79bd132be06d007ce86"
server_key_version: "encryptionkey@0"
I20260806 18:34:20.032608 7668 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:34:20.034996 7695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:20.036177 7668 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:20.036288 7668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-2/wal
uuid: "bed777f10a204fbcb1f8c9724f03c7ae"
format_stamp: "Formatted at 2026-08-06 18:34:20 on dist-test-slave-5bn2"
server_key: "caa5e5d6f48d2f3da02e3e26d3207129"
server_key_iv: "bcaac80ed1aaa79bd132be06d007ce86"
server_key_version: "encryptionkey@0"
I20260806 18:34:20.036391 7668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:20.047303 7668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:20.050381 7668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:20.050568 7668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:20.057523 7668 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.252:42259
I20260806 18:34:20.057838 7775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.252:42259 every 8 connection(s)
I20260806 18:34:20.058574 7668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-2/data/info.pb
I20260806 18:34:20.061067 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 7668
I20260806 18:34:20.061203 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786040973123132-7439-0/minicluster-data/master-2/wal/instance
I20260806 18:34:20.061542 7439 external_mini_cluster.cc:1428] Setting key e08fcffcdea705178a04140cf90a5b03
I20260806 18:34:20.061856 7777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:20.066615 7777 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:34:20 dist-test-slave-5bn2 krb5kdc[7435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.252@KRBTEST.COM
Aug 06 18:34:20 dist-test-slave-5bn2 krb5kdc[7435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.252@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:34:20.076512 7780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:20.062946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54764 (local address 127.7.67.252:42259)
0806 18:34:20.063362 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.063378 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.063415 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:34:20.064105 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.064122 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.064448 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.064731 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.065648 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.066635 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.067792 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.068322 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.068479 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:20.071348 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:20.071353 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:20.071363 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:20.071391 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:20.073322 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.073727 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.073733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.073739 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.073900 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.074225 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.074228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.074230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.074761 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:20.074989 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:20.075235 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 18:34:20.075521 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":125,"threads_started":1}
I20260806 18:34:20.077661 7553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:20.068614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38326 (local address 127.7.67.254:40895)
0806 18:34:20.068783 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.068788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.069093 (+ 305us) server_negotiation.cc:407] Connection header received
0806 18:34:20.069280 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.069283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.069336 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.069410 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.070251 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.070980 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.071892 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.072106 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.072189 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:20.074308 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:20.074311 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:20.074314 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:20.074341 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:20.075948 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.076382 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.076386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.076388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.076475 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.076756 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.076759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.076761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.076956 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:20.077013 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:20.077391 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 18:34:20.077497 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:34:20.077668 7781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:20.068534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:40895 (local address 127.0.0.1:38326)
0806 18:34:20.068954 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:20.069008 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:20.069138 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:20.069480 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:20.069486 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:20.069935 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:20.070127 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:20.070136 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.071100 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.071103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:20.071790 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:20.071799 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.072219 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.072222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:20.072305 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:20.072710 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:20.072725 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:20.074161 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:20.076063 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:20.076072 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:20.076077 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:20.076286 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:20.076555 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:20.076557 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:20.076558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:20.076638 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:20.077096 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:20.077101 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:20.077195 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 18:34:20.077309 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 18:34:20.077483 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":165,"threads_started":1}
I20260806 18:34:20.078970 7777 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:34:20 dist-test-slave-5bn2 krb5kdc[7435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.252@KRBTEST.COM for kudu/127.7.67.253@KRBTEST.COM
Aug 06 18:34:20 dist-test-slave-5bn2 krb5kdc[7435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:34:20.085613 7550 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } attempt: 3
I20260806 18:34:20.088315 7781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:20.079417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:39215 (local address 127.0.0.1:37540)
0806 18:34:20.079524 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:20.079537 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:20.079605 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:20.079885 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:20.079888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:20.080043 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:20.080152 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:20.080159 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.081158 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.081161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:20.081966 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:20.081976 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.082551 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.082555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:20.082650 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:20.083140 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:20.083154 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:20.084483 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:20.086710 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:20.086712 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:20.086715 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:20.086991 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:20.087314 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:20.087318 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:20.087320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:20.087404 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:20.087811 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:20.087820 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:20.087906 (+ 86us) client_negotiation.cc:769] Sending connection context
0806 18:34:20.088004 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 18:34:20.088142 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 18:34:20.088344 7662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:20.079566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37540 (local address 127.7.67.253:39215)
0806 18:34:20.079695 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.079699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.079711 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:34:20.079749 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.079751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.079786 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.079844 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.080268 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.081072 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.082082 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.082455 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.082542 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:20.084635 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:20.084639 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:20.084642 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:20.084693 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:20.086302 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.087102 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.087106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.087108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.087205 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.087518 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.087521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.087523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.087717 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:20.087803 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:20.088068 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 18:34:20.088191 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:34:20.088798 7553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:20.079310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38336 (local address 127.7.67.254:40895)
0806 18:34:20.079418 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.079421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.079434 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:34:20.079469 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.079472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.079513 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.079590 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.080268 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.081031 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.082089 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.082423 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.082519 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:20.085185 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:20.085189 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:20.085192 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:20.085217 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:20.086688 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.087133 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.087136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.087138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.087210 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.087512 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.087515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.087517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.087727 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:20.087959 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:20.088577 (+ 618us) server_negotiation.cc:299] Negotiation successful
0806 18:34:20.088672 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 18:34:20.089362 7777 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:34:20 dist-test-slave-5bn2 krb5kdc[7435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041259, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for kudu/127.7.67.252@KRBTEST.COM
I20260806 18:34:20.094689 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:20.086037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:42259 (local address 127.0.0.1:54772)
0806 18:34:20.086320 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:20.086334 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:20.086412 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:20.086652 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:20.086655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:20.086820 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:20.086956 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:20.086963 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.087981 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.087984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:20.088768 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:20.088780 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.089464 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.089467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:20.089544 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:20.089941 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:20.089955 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:20.091416 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:20.093278 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:20.093281 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:20.093283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:20.093518 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:20.093792 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:20.093795 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:20.093797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:20.093877 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:20.094211 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:20.094215 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:20.094303 (+ 88us) client_negotiation.cc:769] Sending connection context
0806 18:34:20.094393 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 18:34:20.094528 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205}
I20260806 18:34:20.094689 7780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:20.086319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54772 (local address 127.7.67.252:42259)
0806 18:34:20.086484 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.086489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.086500 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:34:20.086530 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.086532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.086563 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.086625 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.087208 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.087876 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.089044 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.089367 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.090204 (+ 837us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:20.091563 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:20.091567 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:20.091570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:20.091592 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:20.093154 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.093631 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.093634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.093636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.093714 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.093960 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.093963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.093965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.094123 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:20.094206 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:20.094452 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 18:34:20.094554 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
Aug 06 18:34:20 dist-test-slave-5bn2 krb5kdc[7435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.252@KRBTEST.COM for kudu/127.7.67.252@KRBTEST.COM
Aug 06 18:34:20 dist-test-slave-5bn2 krb5kdc[7435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.253@KRBTEST.COM
I20260806 18:34:20.098452 7550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578: Bootstrap starting.
I20260806 18:34:20.099258 7662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:20.089976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37546 (local address 127.7.67.253:39215)
0806 18:34:20.090095 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.090099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.090553 (+ 454us) server_negotiation.cc:407] Connection header received
0806 18:34:20.090652 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.090655 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.090697 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.090758 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.091388 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.091889 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.092873 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.093221 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.093313 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:20.096214 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:20.096218 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:20.096221 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:20.096246 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:20.097602 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.097983 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.097987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.097988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.098089 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.098398 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.098401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.098403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.098624 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:20.098837 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:20.098994 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 18:34:20.099110 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:34:20.099635 7781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:20.089809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:42259 (local address 127.0.0.1:54774)
0806 18:34:20.090004 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:20.090017 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:20.090112 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:20.090646 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:20.090649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:20.090800 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:20.090920 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:20.090928 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.092294 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.092298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:20.092997 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:20.093006 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.093458 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.093462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:20.093586 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:20.094006 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:20.094021 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:20.095894 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:20.098003 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:20.098006 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:20.098008 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:20.098228 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:20.098477 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:20.098480 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:20.098482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:20.098573 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:20.099001 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:20.099004 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:20.099093 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 18:34:20.099321 (+ 228us) client_negotiation.cc:240] Negotiation successful
0806 18:34:20.099450 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":19,"spinlock_wait_cycles":3968}
I20260806 18:34:20.099690 7783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:20.090047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54774 (local address 127.7.67.252:42259)
0806 18:34:20.090381 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.090390 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.090401 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:34:20.090428 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.090431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.090469 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.090552 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.091029 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.092176 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.093099 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.093361 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.093453 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:20.096022 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:20.096025 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:20.096028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:20.096051 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:20.097597 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.098309 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.098312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.098314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.098396 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.098657 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.098660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.098662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.098883 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:20.098970 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:20.099397 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 18:34:20.099510 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":273,"thread_start_us":115,"threads_started":1}
I20260806 18:34:20.100073 7658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } attempt: 3
I20260806 18:34:20.101933 7550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:20.102762 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:20.102903 7550 log.cc:824] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:20.103024 7777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae: Bootstrap starting.
I20260806 18:34:20.104300 7513 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "bc92dd2874cc4ed28316da4650831578"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38336
I20260806 18:34:20.105651 7777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:20.106062 7550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578: No bootstrap required, opened a new log
Aug 06 18:34:20 dist-test-slave-5bn2 krb5kdc[7435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041259, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.253@KRBTEST.COM for kudu/127.7.67.252@KRBTEST.COM
I20260806 18:34:20.106508 7777 log.cc:824] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:20.108723 7777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae: No bootstrap required, opened a new log
I20260806 18:34:20.109054 7550 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc92dd2874cc4ed28316da4650831578" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } } peers { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } } peers { permanent_uuid: "bed777f10a204fbcb1f8c9724f03c7ae" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } }
I20260806 18:34:20.109273 7550 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92dd2874cc4ed28316da4650831578, State: Initialized, Role: FOLLOWER
I20260806 18:34:20.109592 7550 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc92dd2874cc4ed28316da4650831578" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } } peers { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } } peers { permanent_uuid: "bed777f10a204fbcb1f8c9724f03c7ae" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } }
I20260806 18:34:20.109972 7783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:20.100899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54780 (local address 127.7.67.252:42259)
0806 18:34:20.101006 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.101010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.101022 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:34:20.101053 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.101055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.101095 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.101611 (+ 516us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.101695 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.102355 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.103402 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.103726 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.104995 (+ 1269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:20.106761 (+ 1766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:20.106765 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:20.106768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:20.106792 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:20.108388 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.108774 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.108778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.108780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.108863 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.109156 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.109160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.109161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.109363 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:20.109479 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:20.109734 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 18:34:20.109836 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:34:20.109972 7661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:20.100435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:42259 (local address 127.0.0.1:54780)
0806 18:34:20.100533 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:20.100547 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:20.100617 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:20.101217 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:20.101220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:20.101386 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:20.101485 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:20.101492 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.102473 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.102477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:20.103250 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:20.103262 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.103865 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.103869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:20.103948 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:20.104434 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:20.104447 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:20.106606 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:20.108488 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:20.108492 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:20.108494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:20.108683 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:20.108980 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:20.108984 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:20.108986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:20.109070 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:20.109487 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:20.109491 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:20.109583 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 18:34:20.109669 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 18:34:20.109807 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 18:34:20.111194 7550 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:20.111387 7513 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 18:34:20.111505 7513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:20.112671 7513 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc92dd2874cc4ed28316da4650831578" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } } peers { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } } peers { permanent_uuid: "bed777f10a204fbcb1f8c9724f03c7ae" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } }
I20260806 18:34:20.112577 7777 raft_consensus.cc:348] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc92dd2874cc4ed28316da4650831578" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } } peers { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } } peers { permanent_uuid: "bed777f10a204fbcb1f8c9724f03c7ae" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } }
I20260806 18:34:20.112811 7777 raft_consensus.cc:729] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bed777f10a204fbcb1f8c9724f03c7ae, State: Initialized, Role: FOLLOWER
I20260806 18:34:20.113121 7658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52: Bootstrap starting.
I20260806 18:34:20.113134 7777 consensus_queue.cc:260] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc92dd2874cc4ed28316da4650831578" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } } peers { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } } peers { permanent_uuid: "bed777f10a204fbcb1f8c9724f03c7ae" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } }
I20260806 18:34:20.113392 7786 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc92dd2874cc4ed28316da4650831578" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } } peers { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } } peers { permanent_uuid: "bed777f10a204fbcb1f8c9724f03c7ae" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } } }
I20260806 18:34:20.113507 7786 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:20.113657 7513 leader_election.cc:288] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc715c5f47a489d9a3e2c25e5ac9d52 (127.7.67.253:39215), bed777f10a204fbcb1f8c9724f03c7ae (127.7.67.252:42259)
I20260806 18:34:20.113739 7801 sys_catalog.cc:455] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc92dd2874cc4ed28316da4650831578" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } } peers { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } } peers { permanent_uuid: "bed777f10a204fbcb1f8c9724f03c7ae" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } } }
I20260806 18:34:20.113924 7801 sys_catalog.cc:458] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:20.114444 7777 sys_catalog.cc:564] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:20.115530 7732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc92dd2874cc4ed28316da4650831578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bed777f10a204fbcb1f8c9724f03c7ae"
I20260806 18:34:20.115737 7732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:20.115890 7658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:20.116736 7658 log.cc:824] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:20.117103 7732 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92dd2874cc4ed28316da4650831578 in term 1.
I20260806 18:34:20.118906 7658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52: No bootstrap required, opened a new log
W20260806 18:34:20.117452 7486 tablet.cc:2369] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:34:20.117841 7479 leader_election.cc:302] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92dd2874cc4ed28316da4650831578, bed777f10a204fbcb1f8c9724f03c7ae; no voters:
I20260806 18:34:20.122217 7786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:20.122411 7786 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [term 1 LEADER]: Becoming Leader. State: Replica: bc92dd2874cc4ed28316da4650831578, State: Running, Role: LEADER
I20260806 18:34:20.122731 7786 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc92dd2874cc4ed28316da4650831578" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } } peers { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } } peers { permanent_uuid: "bed777f10a204fbcb1f8c9724f03c7ae" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } }
I20260806 18:34:20.124450 7786 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc92dd2874cc4ed28316da4650831578. Latest consensus state: current_term: 1 leader_uuid: "bc92dd2874cc4ed28316da4650831578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc92dd2874cc4ed28316da4650831578" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } } peers { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } } peers { permanent_uuid: "bed777f10a204fbcb1f8c9724f03c7ae" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } } }
I20260806 18:34:20.124552 7786 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:20.128165 7658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc92dd2874cc4ed28316da4650831578" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } } peers { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } } peers { permanent_uuid: "bed777f10a204fbcb1f8c9724f03c7ae" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } }
I20260806 18:34:20.128417 7658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc715c5f47a489d9a3e2c25e5ac9d52, State: Initialized, Role: FOLLOWER
I20260806 18:34:20.128758 7658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc92dd2874cc4ed28316da4650831578" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } } peers { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } } peers { permanent_uuid: "bed777f10a204fbcb1f8c9724f03c7ae" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } }
I20260806 18:34:20.129328 7835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc92dd2874cc4ed28316da4650831578" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } } peers { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } } peers { permanent_uuid: "bed777f10a204fbcb1f8c9724f03c7ae" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } } }
I20260806 18:34:20.129590 7835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:20.130245 7658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:20.130314 7617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc92dd2874cc4ed28316da4650831578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52"
I20260806 18:34:20.130540 7617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:20.127580 7461 master_runner.cc:186] Error getting master registration for 127.7.67.253:39215: OK, instance_id { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" instance_seqno: 1786041259846420 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:34:20.130810 7461 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:34:20.132350 7617 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92dd2874cc4ed28316da4650831578 in term 1.
W20260806 18:34:20.120170 7800 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:34:20.132915 7800 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:34:20.134004 7662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:20.119986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37562 (local address 127.7.67.253:39215)
0806 18:34:20.120104 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.120109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.120612 (+ 503us) server_negotiation.cc:407] Connection header received
0806 18:34:20.120809 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.120813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.120870 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.120919 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.121868 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.122548 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.125749 (+ 3201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.126066 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.126142 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:20.127977 (+ 1835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:20.127981 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:20.127984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:20.128009 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:20.131512 (+ 3503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.132021 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.132025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.132027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.132139 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.132440 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.132444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.132445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.132668 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:20.132816 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:20.133734 (+ 918us) server_negotiation.cc:299] Negotiation successful
0806 18:34:20.133857 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:34:20.131078 7668 master_runner.cc:186] Error getting master registration for 127.7.67.253:39215: OK, instance_id { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" instance_seqno: 1786041259846420 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:34:20.134233 7668 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 18:34:20.137986 7820 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:34:20.138078 7820 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:34:20.138571 7834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:20.139600 7834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260806 18:34:20.141471 7848 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:34:20.141556 7848 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:34:20.142575 7553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:20.120720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38350 (local address 127.7.67.254:40895)
0806 18:34:20.120827 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.120831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.133503 (+ 12672us) server_negotiation.cc:407] Connection header received
0806 18:34:20.133675 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.133678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.133725 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.133799 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.134939 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.135606 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.136648 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.136945 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.137021 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:20.138762 (+ 1741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:20.138766 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:20.138769 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:20.138795 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:20.140617 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.141101 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.141104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.141106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.141194 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.141509 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.141512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.141514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.141841 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:20.141943 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:20.142289 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 18:34:20.142421 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:34:20.146126 7732 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae [term 1 FOLLOWER]: Refusing update from remote peer bc92dd2874cc4ed28316da4650831578: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:20.146826 7831 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bed777f10a204fbcb1f8c9724f03c7ae" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:20.147325 7783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:20.121050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54788 (local address 127.7.67.252:42259)
0806 18:34:20.121236 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.121240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.134745 (+ 13505us) server_negotiation.cc:407] Connection header received
0806 18:34:20.134789 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.134792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.134832 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.134934 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.136644 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.137335 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.141684 (+ 4349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.142016 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.142084 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:20.143663 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:20.143667 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:20.143670 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:20.143697 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:20.145297 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.145808 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.145812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.145813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.145906 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.146209 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.146212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.146214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.146452 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:20.146592 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:20.147053 (+ 461us) server_negotiation.cc:299] Negotiation successful
0806 18:34:20.147173 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260806 18:34:20.150868 7801 sys_catalog.cc:455] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc92dd2874cc4ed28316da4650831578. Latest consensus state: current_term: 1 leader_uuid: "bc92dd2874cc4ed28316da4650831578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc92dd2874cc4ed28316da4650831578" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } } peers { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } } peers { permanent_uuid: "bed777f10a204fbcb1f8c9724f03c7ae" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } } }
I20260806 18:34:20.151000 7801 sys_catalog.cc:458] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:20.152784 7617 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52 [term 1 FOLLOWER]: Refusing update from remote peer bc92dd2874cc4ed28316da4650831578: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:20.153357 7831 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:20.156000 7831 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc92dd2874cc4ed28316da4650831578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc92dd2874cc4ed28316da4650831578" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } } peers { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } } peers { permanent_uuid: "bed777f10a204fbcb1f8c9724f03c7ae" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } } }
I20260806 18:34:20.156136 7831 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:20.157106 7801 sys_catalog.cc:455] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc92dd2874cc4ed28316da4650831578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc92dd2874cc4ed28316da4650831578" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } } peers { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } } peers { permanent_uuid: "bed777f10a204fbcb1f8c9724f03c7ae" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } } }
I20260806 18:34:20.157217 7801 sys_catalog.cc:458] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:20.157680 7835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc92dd2874cc4ed28316da4650831578. Latest consensus state: current_term: 1 leader_uuid: "bc92dd2874cc4ed28316da4650831578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc92dd2874cc4ed28316da4650831578" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } } peers { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } } peers { permanent_uuid: "bed777f10a204fbcb1f8c9724f03c7ae" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } } }
I20260806 18:34:20.157793 7835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:20.158296 7852 mvcc.cc:205] Tried to move back new op lower bound from 7315625001547722752 to 7315625001466036224. Current Snapshot: MvccSnapshot[applied={T|T < 7315625001547722752}]
I20260806 18:34:20.158463 7853 mvcc.cc:205] Tried to move back new op lower bound from 7315625001547722752 to 7315625001466036224. Current Snapshot: MvccSnapshot[applied={T|T < 7315625001547722752}]
I20260806 18:34:20.163561 7834 catalog_manager.cc:1347] Generated new cluster ID: 832c7baaa2bc4bceb7931f8ac7504c93
I20260806 18:34:20.163636 7834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:20.164939 7786 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc92dd2874cc4ed28316da4650831578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc92dd2874cc4ed28316da4650831578" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } } peers { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } } peers { permanent_uuid: "bed777f10a204fbcb1f8c9724f03c7ae" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } } }
I20260806 18:34:20.165071 7786 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:20.166190 7835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc92dd2874cc4ed28316da4650831578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc92dd2874cc4ed28316da4650831578" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 40895 } } peers { permanent_uuid: "6fc715c5f47a489d9a3e2c25e5ac9d52" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 39215 } } peers { permanent_uuid: "bed777f10a204fbcb1f8c9724f03c7ae" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 42259 } } }
I20260806 18:34:20.166311 7835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:20.166792 7856 mvcc.cc:205] Tried to move back new op lower bound from 7315625001547722752 to 7315625001466036224. Current Snapshot: MvccSnapshot[applied={T|T < 7315625001547722752}]
I20260806 18:34:20.206971 7834 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:20.211939 7834 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:20.230650 7834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc92dd2874cc4ed28316da4650831578: Generated new TSK 0
I20260806 18:34:20.231833 7834 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:34:20.268254 7662 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:20.264678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37568 (local address 127.7.67.253:39215)
0806 18:34:20.264832 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.264838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.267053 (+ 2215us) server_negotiation.cc:407] Connection header received
0806 18:34:20.267276 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.267280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.267338 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.267391 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.268099 (+ 708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37568: BlockingRecv error: recv got EOF from 127.0.0.1:37568 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 18:34:20.269721 7553 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:20.265685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38360 (local address 127.7.67.254:40895)
0806 18:34:20.265810 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.265814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.268780 (+ 2966us) server_negotiation.cc:407] Connection header received
0806 18:34:20.268954 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.268956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.269008 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.269059 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.269591 (+ 532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38360: BlockingRecv error: recv got EOF from 127.0.0.1:38360 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 18:34:20.271365 7783 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:20.266385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54794 (local address 127.7.67.252:42259)
0806 18:34:20.266504 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.266508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.270216 (+ 3708us) server_negotiation.cc:407] Connection header received
0806 18:34:20.270525 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.270528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.270606 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.270665 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.271236 (+ 571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54794: BlockingRecv error: recv got EOF from 127.0.0.1:54794 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 18:34:20.276484 7662 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:20.274676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37580 (local address 127.7.67.253:39215)
0806 18:34:20.274796 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.274800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.275402 (+ 602us) server_negotiation.cc:407] Connection header received
0806 18:34:20.275668 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.275672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.275759 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.275818 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.276371 (+ 553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37580: BlockingRecv error: recv got EOF from 127.0.0.1:37580 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 18:34:20.281813 7662 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:20.280097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37592 (local address 127.7.67.253:39215)
0806 18:34:20.280204 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.280208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.280796 (+ 588us) server_negotiation.cc:407] Connection header received
0806 18:34:20.281005 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.281009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.281066 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.281122 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.281698 (+ 576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37592: BlockingRecv error: recv got EOF from 127.0.0.1:37592 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 18:34:20.288226 7662 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:20.285957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37606 (local address 127.7.67.253:39215)
0806 18:34:20.286083 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.286087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.287175 (+ 1088us) server_negotiation.cc:407] Connection header received
0806 18:34:20.287393 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.287397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.287453 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.287509 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.288110 (+ 601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37606: BlockingRecv error: recv got EOF from 127.0.0.1:37606 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 18:34:20.295951 7662 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:20.294620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37616 (local address 127.7.67.253:39215)
0806 18:34:20.294781 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.294785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.295042 (+ 257us) server_negotiation.cc:407] Connection header received
0806 18:34:20.295237 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.295240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.295289 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.295335 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.295838 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37616: BlockingRecv error: recv got EOF from 127.0.0.1:37616 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260806 18:34:20.308444 7662 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:20.306710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37618 (local address 127.7.67.253:39215)
0806 18:34:20.306834 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.306839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.307450 (+ 611us) server_negotiation.cc:407] Connection header received
0806 18:34:20.307647 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.307651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.307706 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.307758 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.308319 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37618: BlockingRecv error: recv got EOF from 127.0.0.1:37618 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 18:34:20.328066 7662 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:20.326819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37632 (local address 127.7.67.253:39215)
0806 18:34:20.326980 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.326985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.327244 (+ 259us) server_negotiation.cc:407] Connection header received
0806 18:34:20.327376 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.327379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.327429 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.327498 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.327942 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37632: BlockingRecv error: recv got EOF from 127.0.0.1:37632 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260806 18:34:20.363399 7662 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:20.362042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37640 (local address 127.7.67.253:39215)
0806 18:34:20.362178 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.362182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.362458 (+ 276us) server_negotiation.cc:407] Connection header received
0806 18:34:20.362605 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.362609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.362670 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.362768 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.363275 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37640: BlockingRecv error: recv got EOF from 127.0.0.1:37640 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 18:34:20.431093 7662 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:20.429697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37652 (local address 127.7.67.253:39215)
0806 18:34:20.429840 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.429845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.430018 (+ 173us) server_negotiation.cc:407] Connection header received
0806 18:34:20.430204 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.430208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.430274 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.430355 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.430961 (+ 606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37652: BlockingRecv error: recv got EOF from 127.0.0.1:37652 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 18:34:20.562657 7662 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:20.561602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37666 (local address 127.7.67.253:39215)
0806 18:34:20.561768 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.561774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.561887 (+ 113us) server_negotiation.cc:407] Connection header received
0806 18:34:20.562007 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.562010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.562061 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.562134 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.562528 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37666: BlockingRecv error: recv got EOF from 127.0.0.1:37666 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 18:34:20.822151 7662 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:20.821016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37672 (local address 127.7.67.253:39215)
0806 18:34:20.821170 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.821174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.821238 (+ 64us) server_negotiation.cc:407] Connection header received
0806 18:34:20.821402 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.821405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.821453 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.821528 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.822023 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37672: BlockingRecv error: recv got EOF from 127.0.0.1:37672 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:34:21.139475 7820 catalog_manager.cc:1259] Loaded cluster ID: 832c7baaa2bc4bceb7931f8ac7504c93
I20260806 18:34:21.139580 7820 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae: loading cluster ID for follower catalog manager: success
I20260806 18:34:21.141397 7820 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae: acquiring CA information for follower catalog manager: success
I20260806 18:34:21.142544 7820 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bed777f10a204fbcb1f8c9724f03c7ae: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:34:21.142689 7848 catalog_manager.cc:1259] Loaded cluster ID: 832c7baaa2bc4bceb7931f8ac7504c93
I20260806 18:34:21.142752 7848 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52: loading cluster ID for follower catalog manager: success
I20260806 18:34:21.144316 7848 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52: acquiring CA information for follower catalog manager: success
I20260806 18:34:21.145354 7848 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6fc715c5f47a489d9a3e2c25e5ac9d52: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:34:21.340549 7957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:21.336469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37686 (local address 127.7.67.253:39215)
0806 18:34:21.336719 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:21.336723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:21.336739 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:34:21.336916 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:21.336919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:21.337049 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:21.337151 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:21.337436 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.337984 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.338886 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.339196 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.339292 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:21.339640 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:21.340133 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:21.340244 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:21.340313 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 18:34:21.340373 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":72,"threads_started":1}
I20260806 18:34:21.347203 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:21.343523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54808 (local address 127.7.67.252:42259)
0806 18:34:21.343761 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:21.343766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:21.343781 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:34:21.343975 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:21.343978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:21.344102 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:21.344206 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:21.344427 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.344960 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.345771 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.345987 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.346073 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:21.346372 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:21.346814 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:21.346961 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:21.346995 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 18:34:21.347051 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":69,"threads_started":1}
I20260806 18:34:21.353538 7957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:21.349871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37702 (local address 127.7.67.253:39215)
0806 18:34:21.350006 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:21.350010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:21.350413 (+ 403us) server_negotiation.cc:407] Connection header received
0806 18:34:21.350541 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:21.350544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:21.350592 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:21.350659 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:21.350900 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.351467 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.352374 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.352715 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.352812 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:21.353056 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:21.353138 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:21.353204 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:21.353343 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 18:34:21.353401 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:34:21.354082 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:21.350269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54820 (local address 127.7.67.252:42259)
0806 18:34:21.350373 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:21.350376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:21.350531 (+ 155us) server_negotiation.cc:407] Connection header received
0806 18:34:21.350643 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:21.350646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:21.350691 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:21.350752 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:21.351119 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.351874 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.352969 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.353264 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.353352 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:21.353646 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:21.353722 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:21.353783 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:21.353905 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 18:34:21.353954 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:34:21.354460 7981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:21.350036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38368 (local address 127.7.67.254:40895)
0806 18:34:21.350517 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:21.350522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:21.350546 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:34:21.350645 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:21.350649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:21.350817 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:21.350954 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:21.351191 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.351961 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.352874 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.353159 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.353286 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:21.353590 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:21.354068 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:21.354155 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:21.354246 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 18:34:21.354308 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":78,"threads_started":1}
I20260806 18:34:21.357738 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 7461
I20260806 18:34:21.369320 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 7559
I20260806 18:34:21.380347 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 7668
2026-08-06T18:34:21Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3196 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:21 dist-test-slave-5bn2 krb5kdc[7987](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:21 dist-test-slave-5bn2 krb5kdc[7987](info): set up 2 sockets
Aug 06 18:34:21 dist-test-slave-5bn2 krb5kdc[7987](info): commencing operation
krb5kdc: starting...
W20260806 18:34:22.506629 7439 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/test-user.keytab.
Aug 06 18:34:22 dist-test-slave-5bn2 krb5kdc[7987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
Entry for principal kudu/127.7.67.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab.
I20260806 18:34:22.635602 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34429
--webserver_interface=127.7.67.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:45585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:34429,127.7.67.253:46873,127.7.67.252:38731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:22.749341 8001 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:34:22.749660 8001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:22.749697 8001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:22.749754 8001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:22.753793 8001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:22.753860 8001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:22.753885 8001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:22.753906 8001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:22.753927 8001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:22.759239 8001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:45585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:34429,127.7.67.253:46873,127.7.67.252:38731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.254.keytab
--principal=kudu/127.7.67.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.254:34429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:22.760337 8001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:22.761377 8001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:22.769963 8014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:22.769972 8008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:22.770206 8001 server_base.cc:1034] running on GCE node
W20260806 18:34:22.769997 8009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:22.770963 8001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:22.772089 8001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:22.773270 8001 hybrid_clock.cc:648] HybridClock initialized: now 1786041262773246 us; error 32 us; skew 500 ppm
Aug 06 18:34:22 dist-test-slave-5bn2 krb5kdc[7987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041262, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:22.776245 8001 init.cc:377] Logged in from keytab as kudu/127.7.67.254@KRBTEST.COM (short username kudu)
I20260806 18:34:22.777277 8001 webserver.cc:460] Webserver started at http://127.7.67.254:39245/ using document root <none> and password file <none>
I20260806 18:34:22.777861 8001 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:22.777942 8001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:22.778174 8001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:22.780400 8001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-0/data/instance:
uuid: "24e9163473134e7cacd382dfc2c04ae9"
format_stamp: "Formatted at 2026-08-06 18:34:22 on dist-test-slave-5bn2"
server_key: "08b00a99ad97a7a4851b681bf865ee49"
server_key_iv: "0069e8fad485105729ad6b8f8d0dad83"
server_key_version: "encryptionkey@0"
I20260806 18:34:22.781009 8001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-0/wal/instance:
uuid: "24e9163473134e7cacd382dfc2c04ae9"
format_stamp: "Formatted at 2026-08-06 18:34:22 on dist-test-slave-5bn2"
server_key: "08b00a99ad97a7a4851b681bf865ee49"
server_key_iv: "0069e8fad485105729ad6b8f8d0dad83"
server_key_version: "encryptionkey@0"
I20260806 18:34:22.785051 8001 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:34:22.787472 8026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:22.788717 8001 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:22.788854 8001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-0/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-0/wal
uuid: "24e9163473134e7cacd382dfc2c04ae9"
format_stamp: "Formatted at 2026-08-06 18:34:22 on dist-test-slave-5bn2"
server_key: "08b00a99ad97a7a4851b681bf865ee49"
server_key_iv: "0069e8fad485105729ad6b8f8d0dad83"
server_key_version: "encryptionkey@0"
I20260806 18:34:22.788959 8001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:22.805794 8001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:22.809036 8001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:22.809230 8001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:22.816187 8001 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.254:34429
I20260806 18:34:22.816488 8100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.254:34429 every 8 connection(s)
I20260806 18:34:22.817179 8001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:22.820065 8101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:22.822698 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 8001
I20260806 18:34:22.822827 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-0/wal/instance
I20260806 18:34:22.823195 7439 external_mini_cluster.cc:1428] Setting key 229a20b387bd8d8eaf314231d24fc463
I20260806 18:34:22.824424 8101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.67.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.253@KRBTEST.COM" created.
Aug 06 18:34:22 dist-test-slave-5bn2 krb5kdc[7987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041262, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:34:22.840831 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:22.826877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53378 (local address 127.7.67.254:34429)
0806 18:34:22.827420 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:22.827437 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:22.827550 (+ 113us) server_negotiation.cc:407] Connection header received
0806 18:34:22.828643 (+ 1093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:22.828649 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:22.828919 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:22.829132 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:22.829874 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:22.831374 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:22.832405 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:22.832981 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:22.833159 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:22.836226 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:22.836233 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:22.836248 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:22.836275 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:22.837824 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:22.838349 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:22.838353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:22.838354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:22.838457 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:22.838762 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:22.838764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:22.838765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:22.839170 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:22.839275 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:22.839695 (+ 420us) server_negotiation.cc:299] Negotiation successful
0806 18:34:22.839907 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"spinlock_wait_cycles":32640,"thread_start_us":150,"threads_started":1}
I20260806 18:34:22.840833 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:22.826875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34429 (local address 127.0.0.1:53378)
0806 18:34:22.827382 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:22.827431 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:22.828321 (+ 890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:22.829112 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:22.829116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:22.829449 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:22.829719 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:22.829732 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:22.831526 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:22.831529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:22.832275 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:22.832284 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:22.833149 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:22.833152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:22.833247 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:22.833829 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:22.833845 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:22.836071 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:22.837933 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:22.837944 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:22.837955 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:22.838250 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:22.838541 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:22.838544 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:22.838546 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:22.838685 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:22.839283 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:22.839290 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:22.839449 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 18:34:22.839642 (+ 193us) client_negotiation.cc:240] Negotiation successful
0806 18:34:22.840014 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":35,"thread_start_us":127,"threads_started":1}
I20260806 18:34:22.843169 8101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:34:22.843878 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:22.843646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:46873 (local address 127.0.0.1:49222)
0806 18:34:22.843754 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:22.843822 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.253:46873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 18:34:22.844056 8027 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.67.253:46873: connect: Connection refused (error 111)
I20260806 18:34:22.844837 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:22.844647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:46873 (local address 127.0.0.1:49236)
0806 18:34:22.844743 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:22.844787 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.253:46873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 18:34:22.844957 8101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.253:46873: Network error: Client connection negotiation failed: client connection to 127.7.67.253:46873: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.253.keytab.
Entry for principal kudu/127.7.67.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.253.keytab.
I20260806 18:34:22.851480 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.253:46873
--webserver_interface=127.7.67.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.253.keytab
--principal=kudu/127.7.67.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:45585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:34429,127.7.67.253:46873,127.7.67.252:38731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:34:22.894173 8101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } attempt: 1
I20260806 18:34:22.895153 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:22.894803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:46873 (local address 127.0.0.1:49244)
0806 18:34:22.895004 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:22.895069 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.253:46873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 18:34:22.895843 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:22.895635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:46873 (local address 127.0.0.1:49260)
0806 18:34:22.895736 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:22.895781 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.253:46873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 18:34:22.895991 8101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.253:46873: Network error: Client connection negotiation failed: client connection to 127.7.67.253:46873: connect: Connection refused (error 111)
I20260806 18:34:22.964154 8101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } attempt: 2
W20260806 18:34:22.964746 8109 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
I20260806 18:34:22.965066 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:22.964745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:46873 (local address 127.0.0.1:49262)
0806 18:34:22.964918 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:22.964983 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.253:46873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 18:34:22.965091 8109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:22.965132 8109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:22.965209 8109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 18:34:22.965816 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:22.965600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:46873 (local address 127.0.0.1:49274)
0806 18:34:22.965708 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:22.965754 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.253:46873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 18:34:22.966492 8101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.253:46873: Network error: Client connection negotiation failed: client connection to 127.7.67.253:46873: connect: Connection refused (error 111)
W20260806 18:34:22.969568 8109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:22.969624 8109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:22.969651 8109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:22.969684 8109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:22.969712 8109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:22.974936 8109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:45585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:34429,127.7.67.253:46873,127.7.67.252:38731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.253.keytab
--principal=kudu/127.7.67.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.253:46873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.67.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:22.975992 8109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:22.977007 8109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:22.985576 8118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:22.985563 8121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:22.985569 8124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:22.985867 8109 server_base.cc:1034] running on GCE node
I20260806 18:34:22.986465 8109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:22.987645 8109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:22.988828 8109 hybrid_clock.cc:648] HybridClock initialized: now 1786041262988815 us; error 33 us; skew 500 ppm
Aug 06 18:34:22 dist-test-slave-5bn2 krb5kdc[7987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041262, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:22.991617 8109 init.cc:377] Logged in from keytab as kudu/127.7.67.253@KRBTEST.COM (short username kudu)
I20260806 18:34:22.992545 8109 webserver.cc:460] Webserver started at http://127.7.67.253:35609/ using document root <none> and password file <none>
I20260806 18:34:22.993120 8109 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:22.993191 8109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:22.993399 8109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:22.995515 8109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-1/data/instance:
uuid: "6046c17e7ad74428ada51edad4fbb760"
format_stamp: "Formatted at 2026-08-06 18:34:22 on dist-test-slave-5bn2"
server_key: "317098606862bf03f9263a353a5fa535"
server_key_iv: "f1c67ec032125d22c0195647f0324b2b"
server_key_version: "encryptionkey@0"
I20260806 18:34:22.996061 8109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-1/wal/instance:
uuid: "6046c17e7ad74428ada51edad4fbb760"
format_stamp: "Formatted at 2026-08-06 18:34:22 on dist-test-slave-5bn2"
server_key: "317098606862bf03f9263a353a5fa535"
server_key_iv: "f1c67ec032125d22c0195647f0324b2b"
server_key_version: "encryptionkey@0"
I20260806 18:34:22.999755 8109 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:34:23.002136 8133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:23.003379 8109 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:23.003525 8109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-1/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-1/wal
uuid: "6046c17e7ad74428ada51edad4fbb760"
format_stamp: "Formatted at 2026-08-06 18:34:22 on dist-test-slave-5bn2"
server_key: "317098606862bf03f9263a353a5fa535"
server_key_iv: "f1c67ec032125d22c0195647f0324b2b"
server_key_version: "encryptionkey@0"
I20260806 18:34:23.003628 8109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:23.037242 8109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:23.040212 8109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:23.040400 8109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:23.047436 8109 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.253:46873
I20260806 18:34:23.047719 8211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.253:46873 every 8 connection(s)
I20260806 18:34:23.048453 8109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-1/data/info.pb
I20260806 18:34:23.048686 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 8109
I20260806 18:34:23.048825 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-1/wal/instance
I20260806 18:34:23.049218 7439 external_mini_cluster.cc:1428] Setting key 1b5ab24a42489529d30c101f10758f1f
I20260806 18:34:23.051527 8212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:23.055486 8212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.67.252@KRBTEST.COM; defaulting to no policy
I20260806 18:34:23.057646 8101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.252@KRBTEST.COM" created.
Aug 06 18:34:23 dist-test-slave-5bn2 krb5kdc[7987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041262, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for kudu/127.7.67.253@KRBTEST.COM
Aug 06 18:34:23 dist-test-slave-5bn2 krb5kdc[7987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041262, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.253@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:34:23.070497 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.058034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:46873 (local address 127.0.0.1:49278)
0806 18:34:23.058221 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.058237 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:23.058342 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:23.059883 (+ 1541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:23.059886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:23.060102 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:23.060234 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:23.060242 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.062113 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.062116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:23.062925 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:23.062937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.063673 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.063677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:23.063779 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.064344 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:23.064361 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:23.066121 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:23.068412 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:23.068424 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:23.068427 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:23.068694 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:23.069018 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:23.069021 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:23.069023 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:23.069117 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:23.069908 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:23.069911 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:23.069996 (+ 85us) client_negotiation.cc:769] Sending connection context
0806 18:34:23.070101 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 18:34:23.070266 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260806 18:34:23.070564 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.057458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53386 (local address 127.7.67.254:34429)
0806 18:34:23.057631 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.057636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.058585 (+ 949us) server_negotiation.cc:407] Connection header received
0806 18:34:23.059695 (+ 1110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.059699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.059743 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.059829 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.061075 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.061832 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.062963 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.063321 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.063436 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.066404 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:23.066407 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:23.066409 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:23.066433 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:23.068561 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.069080 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.069084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.069085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.069152 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.069446 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.069449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.069451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.069631 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:23.069716 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:23.070164 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 18:34:23.070350 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":63}
I20260806 18:34:23.071476 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.057891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34429 (local address 127.0.0.1:53386)
0806 18:34:23.058435 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.058489 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:23.059399 (+ 910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:23.059924 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:23.059930 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:23.060361 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:23.060906 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:23.060941 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.061958 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.061961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:23.062825 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:23.062835 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.063522 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.063527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:23.063713 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.064440 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:23.064460 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:23.066280 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:23.068670 (+ 2390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:23.068680 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:23.068692 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:23.068984 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:23.069246 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:23.069248 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:23.069250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:23.069350 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:23.069735 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:23.069742 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:23.069875 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 18:34:23.070088 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 18:34:23.070424 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":39,"thread_start_us":166,"threads_started":1}
I20260806 18:34:23.071477 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.058173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49278 (local address 127.7.67.253:46873)
0806 18:34:23.058598 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.058610 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.058651 (+ 41us) server_negotiation.cc:407] Connection header received
0806 18:34:23.059392 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.059414 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.059714 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.059888 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.060708 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.062010 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.063033 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.063569 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.063716 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.066297 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:23.066306 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:23.066322 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:23.066352 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:23.068286 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.068803 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.068809 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.068810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.068928 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.069210 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.069214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.069216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.069797 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:23.069941 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:23.070408 (+ 467us) server_negotiation.cc:299] Negotiation successful
0806 18:34:23.070557 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":328,"thread_start_us":192,"threads_started":1}
I20260806 18:34:23.072751 8212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:34:23.073539 8101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:34:23.074213 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.073994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:38731 (local address 127.0.0.1:41580)
0806 18:34:23.074100 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.074153 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:38731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 18:34:23.074981 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.074711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:38731 (local address 127.0.0.1:41594)
0806 18:34:23.074869 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.074927 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:38731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 18:34:23.075099 8101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.252:38731: Network error: Client connection negotiation failed: client connection to 127.7.67.252:38731: connect: Connection refused (error 111)
Aug 06 18:34:23 dist-test-slave-5bn2 krb5kdc[7987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041262, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.253@KRBTEST.COM for kudu/127.7.67.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.252.keytab.
Entry for principal kudu/127.7.67.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.252.keytab.
I20260806 18:34:23.079340 7439 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
/tmp/dist-test-taskM6OdK6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.252:38731
--webserver_interface=127.7.67.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.252.keytab
--principal=kudu/127.7.67.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.212:45585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.254:34429,127.7.67.253:46873,127.7.67.252:38731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:34:23.084194 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.073248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:46873 (local address 127.0.0.1:49288)
0806 18:34:23.073584 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.073598 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:23.073687 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:23.073904 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:23.073907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:23.074059 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:23.074154 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:23.074159 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.075139 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.075142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:23.075700 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:23.075709 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.076194 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.076197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:23.076284 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.076734 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:23.076746 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:23.077922 (+ 1176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:23.082755 (+ 4833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:23.082758 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:23.082760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:23.083024 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:23.083298 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:23.083301 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:23.083302 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:23.083379 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:23.083790 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:23.083792 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:23.083875 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 18:34:23.083953 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 18:34:23.084065 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265}
I20260806 18:34:23.084347 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.073280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49288 (local address 127.7.67.253:46873)
0806 18:34:23.073375 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.073379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.073660 (+ 281us) server_negotiation.cc:407] Connection header received
0806 18:34:23.073756 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.073758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.073795 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.073872 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.074261 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.075006 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.075824 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.076094 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.076181 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.078044 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:23.078048 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:23.078049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:23.078098 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:23.082620 (+ 4522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.083121 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.083125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.083127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.083215 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.083483 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.083492 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.083494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.083711 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:23.083804 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:23.084048 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 18:34:23.084173 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 18:34:23.085140 8212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:34:23.085950 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.085649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:38731 (local address 127.0.0.1:41604)
0806 18:34:23.085785 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.085880 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:38731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 18:34:23.086213 8139 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.67.252:38731: connect: Connection refused (error 111)
I20260806 18:34:23.087184 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.086985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:38731 (local address 127.0.0.1:41608)
0806 18:34:23.087086 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.087132 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:38731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 18:34:23.087323 8212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.252:38731: Network error: Client connection negotiation failed: client connection to 127.7.67.252:38731: connect: Connection refused (error 111)
I20260806 18:34:23.106217 8101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } attempt: 1
I20260806 18:34:23.107159 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.106800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:38731 (local address 127.0.0.1:41616)
0806 18:34:23.106985 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.107060 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:38731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 18:34:23.107906 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.107691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:38731 (local address 127.0.0.1:41630)
0806 18:34:23.107799 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.107845 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:38731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 18:34:23.108074 8101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.252:38731: Network error: Client connection negotiation failed: client connection to 127.7.67.252:38731: connect: Connection refused (error 111)
I20260806 18:34:23.136626 8212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } attempt: 1
I20260806 18:34:23.137611 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.137259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:38731 (local address 127.0.0.1:41634)
0806 18:34:23.137447 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.137522 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:38731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 18:34:23.138399 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.138163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:38731 (local address 127.0.0.1:41636)
0806 18:34:23.138274 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.138334 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:38731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 18:34:23.138542 8212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.252:38731: Network error: Client connection negotiation failed: client connection to 127.7.67.252:38731: connect: Connection refused (error 111)
I20260806 18:34:23.183264 8101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } attempt: 2
I20260806 18:34:23.184211 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.183887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:38731 (local address 127.0.0.1:41638)
0806 18:34:23.184054 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.184123 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:38731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 18:34:23.185050 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.184807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:38731 (local address 127.0.0.1:41652)
0806 18:34:23.184939 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.184987 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:38731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 18:34:23.185194 8101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.252:38731: Network error: Client connection negotiation failed: client connection to 127.7.67.252:38731: connect: Connection refused (error 111)
W20260806 18:34:23.196825 8219 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:34:23.197134 8219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:23.197180 8219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:23.197242 8219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:23.201231 8219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:23.201295 8219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:23.201323 8219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:23.201344 8219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:23.201372 8219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:23.206688 8212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } attempt: 2
I20260806 18:34:23.206712 8219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.212:45585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.254:34429,127.7.67.253:46873,127.7.67.252:38731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/krb5kdc/kudu_127.7.67.252.keytab
--principal=kudu/127.7.67.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.252:38731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.67.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:23.207546 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.207234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:38731 (local address 127.0.0.1:41664)
0806 18:34:23.207389 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.207463 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:38731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 18:34:23.208000 8219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:23.208321 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.208106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:38731 (local address 127.0.0.1:41680)
0806 18:34:23.208209 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.208260 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.252:38731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 18:34:23.208469 8212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.252:38731: Network error: Client connection negotiation failed: client connection to 127.7.67.252:38731: connect: Connection refused (error 111)
I20260806 18:34:23.209163 8219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:23.217741 8234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:23.217743 8228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:23.217742 8226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:23.218060 8219 server_base.cc:1034] running on GCE node
I20260806 18:34:23.218650 8219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:23.219786 8219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:23.220979 8219 hybrid_clock.cc:648] HybridClock initialized: now 1786041263220944 us; error 40 us; skew 500 ppm
Aug 06 18:34:23 dist-test-slave-5bn2 krb5kdc[7987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041263, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:23.223848 8219 init.cc:377] Logged in from keytab as kudu/127.7.67.252@KRBTEST.COM (short username kudu)
I20260806 18:34:23.224781 8219 webserver.cc:460] Webserver started at http://127.7.67.252:34119/ using document root <none> and password file <none>
I20260806 18:34:23.225349 8219 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:23.225423 8219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:23.225636 8219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:23.227837 8219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-2/data/instance:
uuid: "6421acff2bae416299b68b96e5a46a66"
format_stamp: "Formatted at 2026-08-06 18:34:23 on dist-test-slave-5bn2"
server_key: "9eab0c9341666e709378f0da36c8bf1e"
server_key_iv: "09d7c4b2153678703081217ff94489c1"
server_key_version: "encryptionkey@0"
I20260806 18:34:23.228415 8219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-2/wal/instance:
uuid: "6421acff2bae416299b68b96e5a46a66"
format_stamp: "Formatted at 2026-08-06 18:34:23 on dist-test-slave-5bn2"
server_key: "9eab0c9341666e709378f0da36c8bf1e"
server_key_iv: "09d7c4b2153678703081217ff94489c1"
server_key_version: "encryptionkey@0"
I20260806 18:34:23.232290 8219 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:34:23.235054 8245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:23.236344 8219 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:34:23.236487 8219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-2/data,/tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-2/wal
uuid: "6421acff2bae416299b68b96e5a46a66"
format_stamp: "Formatted at 2026-08-06 18:34:23 on dist-test-slave-5bn2"
server_key: "9eab0c9341666e709378f0da36c8bf1e"
server_key_iv: "09d7c4b2153678703081217ff94489c1"
server_key_version: "encryptionkey@0"
I20260806 18:34:23.236598 8219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:23.254449 8219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:23.257817 8219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:23.258018 8219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:23.265318 8219 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.252:38731
I20260806 18:34:23.265611 8325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.252:38731 every 8 connection(s)
I20260806 18:34:23.266341 8219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-2/data/info.pb
I20260806 18:34:23.267396 7439 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu as pid 8219
I20260806 18:34:23.267556 7439 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM6OdK6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786040973123132-7439-0/minicluster-data/master-2/wal/instance
I20260806 18:34:23.267894 7439 external_mini_cluster.cc:1428] Setting key b48126b96b4c445ab952daf01ce29534
I20260806 18:34:23.269872 8326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:23.274621 8326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:34:23 dist-test-slave-5bn2 krb5kdc[7987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.252@KRBTEST.COM
Aug 06 18:34:23 dist-test-slave-5bn2 krb5kdc[7987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041263, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.252@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:34:23.283227 8329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.269293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41694 (local address 127.7.67.252:38731)
0806 18:34:23.269718 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.269736 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.269780 (+ 44us) server_negotiation.cc:407] Connection header received
0806 18:34:23.270682 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.270706 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.271167 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.271551 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.272250 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.273751 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.274624 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.275354 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.275523 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.278379 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:23.278387 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:23.278400 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:23.278434 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:23.280237 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.280649 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.280655 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.280661 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.280785 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.281060 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.281063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.281065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.281476 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:23.281665 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:23.281968 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 18:34:23.282246 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":117,"threads_started":1}
I20260806 18:34:23.284340 8101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } attempt: 3
I20260806 18:34:23.286235 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.276741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53394 (local address 127.7.67.254:34429)
0806 18:34:23.276857 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.276861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.277250 (+ 389us) server_negotiation.cc:407] Connection header received
0806 18:34:23.277382 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.277386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.277446 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.277523 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.278411 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.279157 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.280094 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.280330 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.280422 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.282488 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:23.282492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:23.282494 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:23.282525 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:23.284213 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.284743 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.284747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.284749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.284846 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.285144 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.285147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.285148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.285433 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:23.285531 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:23.285955 (+ 424us) server_negotiation.cc:299] Negotiation successful
0806 18:34:23.286077 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:34:23.287086 8330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.276720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.254:34429 (local address 127.0.0.1:53394)
0806 18:34:23.277100 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.277154 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:23.277289 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:23.277594 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:23.277600 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:23.278072 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:23.278259 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:23.278268 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.279279 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.279283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:23.279971 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:23.279981 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.280436 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.280440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:23.280566 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.281014 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:23.281025 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:23.282350 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:23.284327 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:23.284335 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:23.284341 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:23.284618 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:23.284954 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:23.284956 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:23.284958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:23.285058 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:23.285587 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:23.285593 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:23.285723 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 18:34:23.286632 (+ 909us) client_negotiation.cc:240] Negotiation successful
0806 18:34:23.286834 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":130,"threads_started":1}
I20260806 18:34:23.288200 8326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:34:23 dist-test-slave-5bn2 krb5kdc[7987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041262, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.254@KRBTEST.COM for kudu/127.7.67.252@KRBTEST.COM
Aug 06 18:34:23 dist-test-slave-5bn2 krb5kdc[7987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.254@KRBTEST.COM
I20260806 18:34:23.293478 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.284738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:38731 (local address 127.0.0.1:41698)
0806 18:34:23.284846 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.284859 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:23.284933 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:23.285515 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:23.285518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:23.285689 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:23.285785 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:23.285791 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.286757 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.286761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:23.287540 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:23.287549 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.288178 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.288182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:23.288265 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.288714 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:23.288728 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:23.290271 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:23.291895 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:23.291898 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:23.291900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:23.292142 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:23.292427 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:23.292430 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:23.292431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:23.292521 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:23.292920 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:23.292923 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:23.293023 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 18:34:23.293147 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 18:34:23.293290 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":1}
I20260806 18:34:23.293507 8329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.285213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41698 (local address 127.7.67.252:38731)
0806 18:34:23.285320 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.285323 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.285334 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:34:23.285365 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.285367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.285406 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.285491 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.285920 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.286642 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.287632 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.287968 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.288067 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.290409 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:23.290413 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:23.290416 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:23.290439 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:23.291797 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.292239 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.292242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.292244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.292334 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.292601 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.292604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.292606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.292816 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:23.292920 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:23.293227 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 18:34:23.293346 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 18:34:23 dist-test-slave-5bn2 krb5kdc[7987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041263, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.252@KRBTEST.COM for kudu/127.7.67.253@KRBTEST.COM
I20260806 18:34:23.296476 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.285801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53410 (local address 127.7.67.254:34429)
0806 18:34:23.286411 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.286415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.286427 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:34:23.286459 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.286462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.286499 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.286564 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.287494 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.288104 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.289201 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.289452 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.289549 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.292829 (+ 3280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:23.292833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:23.292836 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:23.292857 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:23.294313 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.294787 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.294791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.294793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.294920 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.295238 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.295242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.295243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.295480 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:23.295663 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:23.295858 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 18:34:23.295976 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":114,"threads_started":1}
I20260806 18:34:23.298048 8101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9: Bootstrap starting.
I20260806 18:34:23.299646 8330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.288672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.253:46873 (local address 127.0.0.1:49294)
0806 18:34:23.288766 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.288779 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:23.288845 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:23.289744 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:23.289748 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:23.290194 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:23.290295 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:23.290301 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.291346 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.291350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:23.292184 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:23.292195 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.292699 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.292701 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:23.292776 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.293339 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:23.293353 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:23.295621 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:23.297784 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:23.297786 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:23.297787 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:23.298025 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:23.298544 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:23.298547 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:23.298548 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:23.298651 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:23.299181 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:23.299183 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:23.299265 (+ 82us) client_negotiation.cc:769] Sending connection context
0806 18:34:23.299346 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 18:34:23.299487 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 18:34:23.299607 8212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } attempt: 3
I20260806 18:34:23.299739 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.289392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49294 (local address 127.7.67.253:46873)
0806 18:34:23.289519 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.289524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.289536 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:34:23.289576 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.289579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.289624 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.289708 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.290455 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.291226 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.292278 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.292593 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.292693 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.295823 (+ 3130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:23.295827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:23.295829 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:23.295861 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:23.297655 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.298136 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.298140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.298141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.298240 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.298767 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.298771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.298773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.299039 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:23.299110 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:23.299425 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 18:34:23.299546 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:34:23.300637 8326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:34:23.302637 8101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:23.303627 8101 log.cc:824] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:34:23 dist-test-slave-5bn2 krb5kdc[7987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.253@KRBTEST.COM
Aug 06 18:34:23 dist-test-slave-5bn2 krb5kdc[7987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041262, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.253@KRBTEST.COM for kudu/127.7.67.252@KRBTEST.COM
Aug 06 18:34:23 dist-test-slave-5bn2 krb5kdc[7987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041263, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.252@KRBTEST.COM for kudu/127.7.67.252@KRBTEST.COM
I20260806 18:34:23.307192 8101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9: No bootstrap required, opened a new log
I20260806 18:34:23.309721 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.300037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:38731 (local address 127.0.0.1:41700)
0806 18:34:23.300145 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.300158 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:23.300224 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:23.300767 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:23.300770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:23.300933 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:23.301013 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:23.301019 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.301863 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.301867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:23.302670 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:23.302681 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.303597 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.303601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:23.303680 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.304079 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:23.304093 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:23.305820 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:23.307908 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:23.307911 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:23.307913 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:23.308328 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:23.308632 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:23.308635 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:23.308637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:23.308723 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:23.309185 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:23.309188 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:23.309280 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 18:34:23.309389 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 18:34:23.309542 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":1,"spinlock_wait_cycles":2816}
I20260806 18:34:23.310019 8332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.298809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49306 (local address 127.7.67.253:46873)
0806 18:34:23.299233 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.299242 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.299256 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:34:23.299291 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.299293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.299339 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.299460 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.300260 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.301074 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.302247 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.302803 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.302932 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.305710 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:23.305714 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:23.305717 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:23.305741 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:23.307977 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.308512 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.308516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.308517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.308603 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.308918 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.308921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.308923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.309151 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:23.309335 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:23.309705 (+ 370us) server_negotiation.cc:299] Negotiation successful
0806 18:34:23.309832 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":194,"server-negotiator.queue_time_us":356,"thread_start_us":141,"threads_started":1}
I20260806 18:34:23.310592 8330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.301078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.252:38731 (local address 127.0.0.1:41714)
0806 18:34:23.301177 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:23.301189 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:23.301257 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:23.301912 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:23.301915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:23.302087 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:23.302168 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:23.302174 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.303186 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.303497 (+ 311us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:23.304540 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:23.304553 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.305192 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.305195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:23.305264 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.305671 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:23.305686 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:23.307094 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:23.308982 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:23.308985 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:23.308987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:23.309265 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:23.309567 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:23.309570 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:23.309572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:23.309715 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:23.310129 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:23.310132 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:23.310219 (+ 87us) client_negotiation.cc:769] Sending connection context
0806 18:34:23.310311 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 18:34:23.310428 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 18:34:23.310746 8329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.300373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41700 (local address 127.7.67.252:38731)
0806 18:34:23.300506 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.300510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.300523 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:34:23.300555 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.300558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.300596 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.300663 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.301135 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.301751 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.303058 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.303383 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.303478 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.305970 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:23.305973 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:23.305976 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:23.306001 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:23.307703 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.308452 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.308455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.308457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.308540 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.308814 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.308817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.308819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.309069 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:23.309183 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:23.310437 (+ 1254us) server_negotiation.cc:299] Negotiation successful
0806 18:34:23.310556 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":48,"spinlock_wait_cycles":55040}
I20260806 18:34:23.311088 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.301229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41714 (local address 127.7.67.252:38731)
0806 18:34:23.301627 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.301635 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.301647 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:34:23.301680 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.301683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.301719 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.301802 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.302274 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.303098 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.304668 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.305097 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.305192 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.307229 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:23.307234 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:23.307237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:23.307262 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:23.308891 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.309348 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.309351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.309352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.309431 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.309803 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.309806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.309808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.309988 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:23.310095 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:23.310795 (+ 700us) server_negotiation.cc:299] Negotiation successful
0806 18:34:23.310942 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":333,"thread_start_us":178,"threads_started":1}
I20260806 18:34:23.313931 7439 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:23.314040 8212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760: Bootstrap starting.
I20260806 18:34:23.315091 8061 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "24e9163473134e7cacd382dfc2c04ae9"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53410
I20260806 18:34:23.315775 8101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e9163473134e7cacd382dfc2c04ae9" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } } peers { permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } } peers { permanent_uuid: "6421acff2bae416299b68b96e5a46a66" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } }
I20260806 18:34:23.316033 8101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24e9163473134e7cacd382dfc2c04ae9, State: Initialized, Role: FOLLOWER
I20260806 18:34:23.316380 8101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e9163473134e7cacd382dfc2c04ae9" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } } peers { permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } } peers { permanent_uuid: "6421acff2bae416299b68b96e5a46a66" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } }
I20260806 18:34:23.316865 8212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:23.316984 8339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e9163473134e7cacd382dfc2c04ae9" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } } peers { permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } } peers { permanent_uuid: "6421acff2bae416299b68b96e5a46a66" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } } }
I20260806 18:34:23.317193 8339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:23.317780 8212 log.cc:824] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:23.317863 8101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:23.317906 8061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 18:34:23.318025 8061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:23.318015 8326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66: Bootstrap starting.
I20260806 18:34:23.319420 8061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e9163473134e7cacd382dfc2c04ae9" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } } peers { permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } } peers { permanent_uuid: "6421acff2bae416299b68b96e5a46a66" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } }
I20260806 18:34:23.320382 8061 leader_election.cc:288] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [CANDIDATE]: Term 1 election: Requested vote from peers 6046c17e7ad74428ada51edad4fbb760 (127.7.67.253:46873), 6421acff2bae416299b68b96e5a46a66 (127.7.67.252:38731)
I20260806 18:34:23.320614 8326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:23.321462 8326 log.cc:824] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66: Log is configured to *not* fsync() on all Append() calls
W20260806 18:34:23.323750 8365 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:34:23.323872 8365 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:34:23.325598 8326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66: No bootstrap required, opened a new log
I20260806 18:34:23.326017 8001 master_runner.cc:186] Error getting master registration for 127.7.67.253:46873: OK, instance_id { permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" instance_seqno: 1786041263045455 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:34:23.329727 8001 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:34:23.329890 8326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e9163473134e7cacd382dfc2c04ae9" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } } peers { permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } } peers { permanent_uuid: "6421acff2bae416299b68b96e5a46a66" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } }
I20260806 18:34:23.330199 8326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6421acff2bae416299b68b96e5a46a66, State: Initialized, Role: FOLLOWER
I20260806 18:34:23.330574 8326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e9163473134e7cacd382dfc2c04ae9" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } } peers { permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } } peers { permanent_uuid: "6421acff2bae416299b68b96e5a46a66" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } }
I20260806 18:34:23.332219 8326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:23.332250 8284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "24e9163473134e7cacd382dfc2c04ae9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6421acff2bae416299b68b96e5a46a66"
I20260806 18:34:23.332464 8284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:23.334419 8284 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e9163473134e7cacd382dfc2c04ae9 in term 1.
I20260806 18:34:23.334941 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.323573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53426 (local address 127.7.67.254:34429)
0806 18:34:23.323684 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.323688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.325105 (+ 1417us) server_negotiation.cc:407] Connection header received
0806 18:34:23.325270 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.325273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.325320 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.325367 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.326392 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.327224 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.328626 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.328966 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.329049 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.331083 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:23.331087 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:23.331091 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:23.331120 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:23.332834 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.333375 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.333378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.333381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.333491 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.333816 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.333821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.333823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.334054 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:23.334171 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:23.334576 (+ 405us) server_negotiation.cc:299] Negotiation successful
0806 18:34:23.334712 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:34:23.334983 8030 leader_election.cc:302] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e9163473134e7cacd382dfc2c04ae9, 6421acff2bae416299b68b96e5a46a66; no voters:
I20260806 18:34:23.335342 8339 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:23.335563 8339 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [term 1 LEADER]: Becoming Leader. State: Replica: 24e9163473134e7cacd382dfc2c04ae9, State: Running, Role: LEADER
I20260806 18:34:23.335950 8339 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e9163473134e7cacd382dfc2c04ae9" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } } peers { permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } } peers { permanent_uuid: "6421acff2bae416299b68b96e5a46a66" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } }
I20260806 18:34:23.340073 8393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24e9163473134e7cacd382dfc2c04ae9. Latest consensus state: current_term: 1 leader_uuid: "24e9163473134e7cacd382dfc2c04ae9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e9163473134e7cacd382dfc2c04ae9" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } } peers { permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } } peers { permanent_uuid: "6421acff2bae416299b68b96e5a46a66" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } } }
I20260806 18:34:23.340543 8393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:23.343576 8212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760: No bootstrap required, opened a new log
I20260806 18:34:23.347618 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.323377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41730 (local address 127.7.67.252:38731)
0806 18:34:23.323647 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.323653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.324797 (+ 1144us) server_negotiation.cc:407] Connection header received
0806 18:34:23.324947 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.324950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.325005 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.325075 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.338180 (+ 13105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.339009 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.340277 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.340648 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.340726 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.343907 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:23.343912 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:23.343914 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:23.343947 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:23.345574 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.346095 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.346099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.346101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.346195 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.346501 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.346504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.346506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.346747 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:23.346934 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:23.347309 (+ 375us) server_negotiation.cc:299] Negotiation successful
0806 18:34:23.347454 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189}
I20260806 18:34:23.348146 8212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e9163473134e7cacd382dfc2c04ae9" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } } peers { permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } } peers { permanent_uuid: "6421acff2bae416299b68b96e5a46a66" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } }
I20260806 18:34:23.348421 8212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6046c17e7ad74428ada51edad4fbb760, State: Initialized, Role: FOLLOWER
I20260806 18:34:23.348784 8332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:23.324197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49314 (local address 127.7.67.253:46873)
0806 18:34:23.324310 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.324315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.331417 (+ 7102us) server_negotiation.cc:407] Connection header received
0806 18:34:23.331775 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.331779 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.331835 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.331888 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.338522 (+ 6634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.339279 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.340301 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:23.340657 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:23.340725 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:23.343436 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:23.343440 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:23.343443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:23.343470 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:23.345700 (+ 2230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.346404 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.346408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.346410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.346508 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:23.347204 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:23.347208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:23.347209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:23.347427 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:23.347556 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:23.348503 (+ 947us) server_negotiation.cc:299] Negotiation successful
0806 18:34:23.348631 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:34:23.348795 8212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e9163473134e7cacd382dfc2c04ae9" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } } peers { permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } } peers { permanent_uuid: "6421acff2bae416299b68b96e5a46a66" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } }
I20260806 18:34:23.350443 8212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:23.350628 8173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "24e9163473134e7cacd382dfc2c04ae9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6046c17e7ad74428ada51edad4fbb760"
I20260806 18:34:23.350945 8173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:23.351162 8395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:23.352183 8395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:23.352375 8173 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e9163473134e7cacd382dfc2c04ae9 in term 1.
I20260806 18:34:23.352470 8380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e9163473134e7cacd382dfc2c04ae9" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } } peers { permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } } peers { permanent_uuid: "6421acff2bae416299b68b96e5a46a66" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } } }
I20260806 18:34:23.352625 8380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:23.353132 8396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e9163473134e7cacd382dfc2c04ae9" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } } peers { permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } } peers { permanent_uuid: "6421acff2bae416299b68b96e5a46a66" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } } }
I20260806 18:34:23.353376 8396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 18:34:23.358321 8412 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:34:23.358531 8412 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 18:34:23.362035 8410 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:34:23.362115 8410 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:34:23.371340 8284 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66 [term 1 FOLLOWER]: Refusing update from remote peer 24e9163473134e7cacd382dfc2c04ae9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:23.372349 8393 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6421acff2bae416299b68b96e5a46a66" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:23.374130 8173 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760 [term 1 FOLLOWER]: Refusing update from remote peer 24e9163473134e7cacd382dfc2c04ae9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:23.374840 8393 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:23.378535 8380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24e9163473134e7cacd382dfc2c04ae9. Latest consensus state: current_term: 1 leader_uuid: "24e9163473134e7cacd382dfc2c04ae9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e9163473134e7cacd382dfc2c04ae9" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } } peers { permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } } peers { permanent_uuid: "6421acff2bae416299b68b96e5a46a66" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } } }
I20260806 18:34:23.378702 8380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:23.379798 8396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24e9163473134e7cacd382dfc2c04ae9. Latest consensus state: current_term: 1 leader_uuid: "24e9163473134e7cacd382dfc2c04ae9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e9163473134e7cacd382dfc2c04ae9" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } } peers { permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } } peers { permanent_uuid: "6421acff2bae416299b68b96e5a46a66" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } } }
I20260806 18:34:23.379933 8396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:23.388016 8396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "24e9163473134e7cacd382dfc2c04ae9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e9163473134e7cacd382dfc2c04ae9" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } } peers { permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } } peers { permanent_uuid: "6421acff2bae416299b68b96e5a46a66" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } } }
I20260806 18:34:23.388188 8396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:23.389122 8393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "24e9163473134e7cacd382dfc2c04ae9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e9163473134e7cacd382dfc2c04ae9" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } } peers { permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } } peers { permanent_uuid: "6421acff2bae416299b68b96e5a46a66" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } } }
I20260806 18:34:23.389271 8393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:23.389601 8413 mvcc.cc:205] Tried to move back new op lower bound from 7315625014758875136 to 7315625014627966976. Current Snapshot: MvccSnapshot[applied={T|T < 7315625014758875136}]
I20260806 18:34:23.389770 8424 mvcc.cc:205] Tried to move back new op lower bound from 7315625014758875136 to 7315625014627966976. Current Snapshot: MvccSnapshot[applied={T|T < 7315625014758875136}]
I20260806 18:34:23.393692 8339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "24e9163473134e7cacd382dfc2c04ae9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e9163473134e7cacd382dfc2c04ae9" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } } peers { permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } } peers { permanent_uuid: "6421acff2bae416299b68b96e5a46a66" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } } }
I20260806 18:34:23.393838 8339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:23.395026 8380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "24e9163473134e7cacd382dfc2c04ae9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24e9163473134e7cacd382dfc2c04ae9" member_type: VOTER last_known_addr { host: "127.7.67.254" port: 34429 } } peers { permanent_uuid: "6046c17e7ad74428ada51edad4fbb760" member_type: VOTER last_known_addr { host: "127.7.67.253" port: 46873 } } peers { permanent_uuid: "6421acff2bae416299b68b96e5a46a66" member_type: VOTER last_known_addr { host: "127.7.67.252" port: 38731 } } }
I20260806 18:34:23.395169 8380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:23.395481 8420 mvcc.cc:205] Tried to move back new op lower bound from 7315625014758875136 to 7315625014627966976. Current Snapshot: MvccSnapshot[applied={T|T < 7315625014758875136}]
I20260806 18:34:23.397260 8395 catalog_manager.cc:1347] Generated new cluster ID: 0c194712d3fc4996857fadc8c36e75c7
I20260806 18:34:23.397338 8395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:23.428464 8395 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:23.429919 8395 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:23.451435 8395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24e9163473134e7cacd382dfc2c04ae9: Generated new TSK 0
I20260806 18:34:23.452596 8395 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:34:23.484544 8334 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:23.481044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41740 (local address 127.7.67.252:38731)
0806 18:34:23.481183 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.481189 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.483234 (+ 2045us) server_negotiation.cc:407] Connection header received
0806 18:34:23.483515 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.483519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.483580 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.483636 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.484381 (+ 745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41740: BlockingRecv error: recv got EOF from 127.0.0.1:41740 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 18:34:23.486107 8106 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:23.481906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53432 (local address 127.7.67.254:34429)
0806 18:34:23.482026 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.482031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.485107 (+ 3076us) server_negotiation.cc:407] Connection header received
0806 18:34:23.485294 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.485297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.485350 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.485416 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.485965 (+ 549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53432: BlockingRecv error: recv got EOF from 127.0.0.1:53432 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 18:34:23.487604 8332 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:23.482588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49326 (local address 127.7.67.253:46873)
0806 18:34:23.482706 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.482710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.486548 (+ 3838us) server_negotiation.cc:407] Connection header received
0806 18:34:23.486752 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.486756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.486811 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.486922 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.487482 (+ 560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49326: BlockingRecv error: recv got EOF from 127.0.0.1:49326 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 18:34:23.492470 8332 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:23.490627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49336 (local address 127.7.67.253:46873)
0806 18:34:23.490745 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.490749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.491337 (+ 588us) server_negotiation.cc:407] Connection header received
0806 18:34:23.491580 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.491583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.491656 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.491709 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.492336 (+ 627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49336: BlockingRecv error: recv got EOF from 127.0.0.1:49336 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 18:34:23.498096 8332 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:23.496395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49346 (local address 127.7.67.253:46873)
0806 18:34:23.496509 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.496513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.497046 (+ 533us) server_negotiation.cc:407] Connection header received
0806 18:34:23.497298 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.497301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.497362 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.497453 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.497989 (+ 536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49346: BlockingRecv error: recv got EOF from 127.0.0.1:49346 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 18:34:23.504246 8332 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:23.502635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49354 (local address 127.7.67.253:46873)
0806 18:34:23.502743 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.502747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.503269 (+ 522us) server_negotiation.cc:407] Connection header received
0806 18:34:23.503482 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.503485 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.503539 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.503594 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.504128 (+ 534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49354: BlockingRecv error: recv got EOF from 127.0.0.1:49354 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 18:34:23.512940 8332 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:23.510951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49358 (local address 127.7.67.253:46873)
0806 18:34:23.511099 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.511104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.511861 (+ 757us) server_negotiation.cc:407] Connection header received
0806 18:34:23.512085 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.512089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.512149 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.512201 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.512812 (+ 611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49358: BlockingRecv error: recv got EOF from 127.0.0.1:49358 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 18:34:23.525956 8332 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:23.523951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49364 (local address 127.7.67.253:46873)
0806 18:34:23.524087 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.524092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.524759 (+ 667us) server_negotiation.cc:407] Connection header received
0806 18:34:23.525010 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.525014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.525078 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.525139 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.525816 (+ 677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49364: BlockingRecv error: recv got EOF from 127.0.0.1:49364 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 18:34:23.546739 8332 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:23.545009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49378 (local address 127.7.67.253:46873)
0806 18:34:23.545144 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.545149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.545789 (+ 640us) server_negotiation.cc:407] Connection header received
0806 18:34:23.545946 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.545949 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.546010 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.546097 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.546611 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49378: BlockingRecv error: recv got EOF from 127.0.0.1:49378 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 18:34:23.582455 8332 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:23.581213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49386 (local address 127.7.67.253:46873)
0806 18:34:23.581373 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.581379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.581540 (+ 161us) server_negotiation.cc:407] Connection header received
0806 18:34:23.581696 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.581699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.581750 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.581822 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.582309 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49386: BlockingRecv error: recv got EOF from 127.0.0.1:49386 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 18:34:23.649844 8332 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:23.648618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49390 (local address 127.7.67.253:46873)
0806 18:34:23.648750 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.648755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.648947 (+ 192us) server_negotiation.cc:407] Connection header received
0806 18:34:23.649111 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.649114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.649164 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.649229 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.649713 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49390: BlockingRecv error: recv got EOF from 127.0.0.1:49390 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 18:34:23.781275 8332 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:23.780046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49392 (local address 127.7.67.253:46873)
0806 18:34:23.780182 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:23.780188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:23.780361 (+ 173us) server_negotiation.cc:407] Connection header received
0806 18:34:23.780523 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:23.780526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:23.780577 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:23.780649 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:23.781142 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49392: BlockingRecv error: recv got EOF from 127.0.0.1:49392 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 18:34:24.040612 8332 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:24.039487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49396 (local address 127.7.67.253:46873)
0806 18:34:24.039633 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:24.039637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:24.039661 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:34:24.039797 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:24.039801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:24.039870 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:24.039955 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:24.040492 (+ 537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49396: BlockingRecv error: recv got EOF from 127.0.0.1:49396 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:34:24.360028 8412 catalog_manager.cc:1259] Loaded cluster ID: 0c194712d3fc4996857fadc8c36e75c7
I20260806 18:34:24.360116 8412 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66: loading cluster ID for follower catalog manager: success
I20260806 18:34:24.361840 8412 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66: acquiring CA information for follower catalog manager: success
I20260806 18:34:24.363389 8412 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6421acff2bae416299b68b96e5a46a66: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:34:24.363468 8410 catalog_manager.cc:1259] Loaded cluster ID: 0c194712d3fc4996857fadc8c36e75c7
I20260806 18:34:24.363516 8410 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760: loading cluster ID for follower catalog manager: success
I20260806 18:34:24.365005 8410 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760: acquiring CA information for follower catalog manager: success
I20260806 18:34:24.366065 8410 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6046c17e7ad74428ada51edad4fbb760: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:34:24.558359 8528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:24.554670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49412 (local address 127.7.67.253:46873)
0806 18:34:24.554994 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:24.554998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:24.555015 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:34:24.555131 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:24.555133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:24.555273 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:24.555376 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:24.555672 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.556206 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.557012 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.557231 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.557304 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:24.557558 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:24.558010 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:24.558106 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:24.558135 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 18:34:24.558188 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":71,"threads_started":1}
I20260806 18:34:24.565412 8537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:24.561669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41746 (local address 127.7.67.252:38731)
0806 18:34:24.561917 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:24.561922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:24.561934 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:34:24.562019 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:24.562021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:24.562139 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:24.562242 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:24.562470 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.563117 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.563929 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.564164 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.564254 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:24.564559 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:24.565006 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:24.565120 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:24.565186 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 18:34:24.565251 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":64,"threads_started":1}
I20260806 18:34:24.572356 8537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:24.568562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41760 (local address 127.7.67.252:38731)
0806 18:34:24.568697 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:24.568701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:24.568958 (+ 257us) server_negotiation.cc:407] Connection header received
0806 18:34:24.569057 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:24.569060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:24.569114 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:24.569182 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:24.569438 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.570137 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.571239 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.571572 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.571665 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:24.571876 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:24.571956 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:24.572016 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:24.572148 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 18:34:24.572211 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:34:24.572836 8528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:24.569024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49422 (local address 127.7.67.253:46873)
0806 18:34:24.569161 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:24.569165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:24.569372 (+ 207us) server_negotiation.cc:407] Connection header received
0806 18:34:24.569583 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:24.569587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:24.569648 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:24.569721 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:24.569940 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.570633 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.571628 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.571932 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.572035 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:24.572347 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:24.572442 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:24.572494 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:24.572653 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 18:34:24.572713 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:34:24.573335 8549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:24.568502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53448 (local address 127.7.67.254:34429)
0806 18:34:24.568744 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:24.568748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:24.568978 (+ 230us) server_negotiation.cc:407] Connection header received
0806 18:34:24.569087 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:24.569091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:24.569262 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:24.569397 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:24.569642 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.570421 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.571466 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.571775 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.571866 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:24.572330 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:24.572708 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:24.572789 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:24.572879 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 18:34:24.573194 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":61,"threads_started":1}
I20260806 18:34:24.576416 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 8001
I20260806 18:34:24.591904 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 8109
I20260806 18:34:24.604374 7439 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM6OdK6/build/debug/bin/kudu with pid 8219
2026-08-06T18:34:24Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3224 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6421 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (291477 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST